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ABSTRACT 


This  final  report  describes  a continuing  effort  to  develop  coherent 
optical  processing  techniques  applicable  to  parallax  measurement.  The 
IMF  system,  an  experimental  coherent  optical  processor,  was  modified 
to  automate  the  parallax  measurement  process.  A CID  camera  system 
was  procured  and  installed  in  the  experimental  system.  Used  with  phase, 
rather  than  amplitude,  matched  filters,  the  CID  camera  provides  adequate 
sensitivity  for  correlation  detection  with  lower  video  noise  levels  and 
a fixed  metric  field.  An  electronic  interface,  capable  of  automatic 
correlation  peak  detection,  was  designed,  fabricated,  and  tested.  This 
interface  and  the  associated  control  software  was  consistent  with  parallax 
data  collection  rates  of  30  samples  per  second.  A preliminary  evaluation 
of  the  IMF  system  for  pattern  recognition  was  performed.  Differences 
in  the  power  spectral  density  were  observed  for  urban  and  rural  regions. 
The  discrimination  provided  by  a simple  feature  (power  spectral  density) 
merits  continued  investigation  (a  concentrated  study  to  evaluate  the 
effectiveness  of  coherent  optical  techniques  for  feature  extraction  and 
pattern  recognition  is  warranted). 


PREFACE 


Thia  report  was  prepared  by  the  Electro -Optica  Department  of  the 
Harria  Government  Communication  Syatema  Division,  Melbourne,  Florida, 
under  Contract  F30602-76-C-0381  with  Rome  Air  Development  Center, 

IRRE,  Griffiaa  Air  Force  Baae,  Rome,  New  York;  the  effort  waa  moni- 
tored by  A.  Pi  rich  of  RADC. 

The  Principal  Inveatigator  and  Program  Manager  waa  F.  B.  Rotz, 
who  reporta  to  A.  Vander  Lugt,  Director  of  the  Electro -Optica  Department. 
The  major  contributor  to  thia  report  are  M.  W.  Shareck  and  F.  B.  Rotz; 
K.  R.  Porter  aaaiated  in  the  experimental  effort. 


EVALUATION 


This  report  summarizes  the  findings  of  a continuing 
program  to  determine  and  provide  improvements  to  an  Image- 
Matched  Filter  (IMF)  Correlator  System. 

The  current  phase  of  development  has  been  primarily 
concerned  with  three  areas: 

a.  The  identification  of  a solid-state  imaging  device 
for  correlation  detection. 

b.  Implementation  of  an  electronic  interface  for 
automatic  correlation  location. 

c.  Development  of  software  for  parallax  collection  at 
30  samples  per  second. 

Standard  vidicons,  charge-coupled  devices,  and  charge- 
injection  devices  were  evaluated  for  possible  use  as  an 
output  correlation  detector.  The  charge-injection  device 
(CID)  was  used  and  provided  adequate  sensitivity  for 
correlation  detection  with  low  video  noise  levels  and  a 
fixed  metric  field. 

Control  Software  for  the  PDP-8A  computer  was  developed 


for  IMF  system  operation  at  a rate  of  30  samples  per  second. 
Computer  routines  to  provide  galvonometer  control,  aperture 


location  and  movement,  and  correlation  peak  detection  were 
written  for  the  programmable  control  unit.  The  investigation 
into  the  possibility  of  employing  the  IMF  system  for  pattern 
recognition  and  feature  extraction  was  rather  weak.  The 
report  definitely  does  not  indicate  a full  appreciation  of 
feature  extraction/pattern  recognition  technology.  Sorting 
the  differences  of  the  Power  Spectrum  from  a few  aerial 
scenes  does  not  constitute  a feature  extraction/pattern 
recognition  investigation.  A portion  of  this  effort  was 
to  explore  the  facets  of  optical  versus  digital  image 
processing  by  employing  the  IMF  Correlation  System.  This 
was  not  fully  accomplished  nor  discussed  within  this  report. 

Comparison  of  the  feature  extraction  and  pattern  recog- 
nition capabilities  between  digital  and  coherent  optical 
systems  is  warranted. 


ANDREW  R.  PIRICH 
Project  Engineer 


SECTION  I 
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INTRODUCTION 

The  extraction  of  elevation  data  from  stereoscopic  aerial  photographs 
requires  determining  with  high  accuracy  the  parallax  associated  with 
distinct  terrain  regions.  This  parallax,  generated  byr  a change  in  viewing 
angle,  is  the  relative  displacement  of  an  object  in  the  two  stereo  scenes. 
Originally,  the  parallax  was  measured  by  manual  stereo-plotters  and 
stereo-comparitors  and  subsequently  reduced  to  obtain  elevation  and  con- 
tour information.  Over  the  past  two  decades,  several  automatic  stereo- 
compilation systems  have  been  developed  utilizing  modern  electronic 
techniques.  In  general,  these  systems  rely  on  a one -dimensional  correl- 
ation scheme.  In  this  scheme,  the  aerial  photographs  comprising  the  stereo 
pair  are  scanned  with  a small  beam;  the  video  signals  resulting  from  the 
scanning  process  are  electronically  cross -correlated  to  determine  the 
parallax. 

Although  these  electronic  correlation  systems  have  been  increasingly 
successful,  improvements  in  operational  performance,  speeds,  and  costs 
are  possible.  The  Rome  Air  Development  Center  (RA DC)  has  been  instru- 
mental in  exploring  alternative  approaches  for  stereocompilation  which 
avoid  the  limitations  of  the  electronic  schemes.  In  particular,  coherent 
optical  data  processing  techniques  offer  advantages  for  both  parallax 
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measurement  and  feature  extraction.  Thia  final  report  describes  a contin- 
uing effort  to  develop  coherent  optical  processing  techniques  applicable  to 

parallax  measurement. 

A key  advantage  of  a coherent  optical  system  is  the  direct  presence  of 
both  an  input  scene  and  its  Fourier  transform  within  the  same  processor. 
Thus,  either  the  image  or  its  transform  can  be  accessed.  In  addition,  the 
system  can  perform  a two  dimensional  cross-correlation  between  an  input 
scene  and  a stored  reference  pattern. 

Under  a previous  contractual  phase,  RADC  assembled  an  Image  Match- 
ed Filter  (IMF)  system,  a large  aperture  coherent  optical  processor  con- 
taining the  components  necessary  for  parallax  measurement.  On  this 
lyitem,  the  basic  suitability  of  optical  processing  techniques  for  parallax 
measurement  was  demonstrated  in  a manual  mode  by  the  Electro-Optics 
Department  of  Harris  Government  Communication  Systems  Division. 
Subsequent  effort  was  directed  to  refine  and  improve  the  breadboard  system. 
During  this  phase,  the  complexity  of  the  optical  system  was  significantly  re- 
duced without  sacrificing  performance,  scanning  accuracy  was  improved, 
and  automatic  techniques  for  scanner  calibration  were  demonstrated.  In 
addition,  preprocessing  techniques  developed  for  the  input  images  increased 
the  uniformity  of  the  correlation  process. 
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The  work  described  in  this  report  is  a continued  e-fort  to  automate  and 
develop  various  techniques  for  processing  aerial  imager/  with  the  IMF 
s/stem.  The  specific  objectives  for  this  contract  (F30602-76-0381)  were: 

• Evaluate  sensors,  including  CCD,  CID,  and  TV  raster  scanners,  to 
determine  the  optimum  correlation  detection  device. 

• Develop  an  experimental  s/stem  capable  of  collecting  parallax  data 
from  vertical  frame  photography  at  rates  of  30  samples  per  second, 
and 

• Provide  a methodology  to  extract  cultural  and  terrain  feature*  from 
aerial  photographs  with  the  IMF  system. 

This  report  details  the  activities  performed  to  complete  these  object- 
ives. Our  results  and  conclusions,  along  with  recommendations  for  future 
activities,  are  summarized  in  Section  II.  A basic  review  of  the  theory 
of  optical  data  processing  is  presented  in  Section  III;  a discussion  of  the 
potential  advantages  offered  by  a coherent  optical  system  for  stereocompilation 
and  feature  extraction  is  also  included.  The  capabilities  and  performance 
offered  by  various  area  detection  devices  are  summarized  in  Section  IV;  the 
optical  and  electronic  system  operations  are  described,  as  well  as  the  software 
control  functions.  A description  of  a technique  applicable  to  feature 
extraction  is  presented  in  Section  VI. 
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SECTION  II 
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RESULTS  AND  CONCLUSIONS 

/ 

In  this  section,  we  briefly  review  the  results  and  conclusions  of  our 
investigation  and  present  our  recommendations  for  future  activity.  Relev- 
ant supporting  material  is  contained  in  subsequent  sections  of  this  report.  A 
complete  discussion  of  solid  state  image  detectors  is  given  in  Section  IV, 
while  Section  V describes  the  present  developmental  status  of  the  IMF  sys- 
tem. The  potential  of  a coherent  optical  processing  system  for  feature 
extraction  is  detailed  in  Section  VI. 

2.1  SUMMARY  OF  RESULTS 

Previous  experimental  evaluation  of  the  IMF  system  has  demonstrated 
the  effectiveness  of  coherent  optical  processing  for  stereocompilation; 
parallax  data  obtained  on  the  IMF  system  showed  good  agreement  with  cor- 
responding data  from  the  AS-UB  system  (Final  Technical  Report  for  Contr- 
act F30602  - 73  - C - 0312).  Additional  contract  effort  was  initiated  to  im- 
prove several  areas  of  concern.  In  this  continued  activity,  the  complexity 
of  the  optical  system  was  reduced  significantly,  the  stability  and  accuracy  of 
the  scanning  system  was  improved,  the  correlation  performance  of  input 
imagery  was  increased,  and  real-time  spatial  filter  recording  materials 
were  investigated.  A complete  discussion  of  these  results  are  contained  in 
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the  Final  Technical  Report  for  Contract  F30602  - 75  - 0305. 

The  current  phase  of  system  development  has  been  primarily  concerned 
with  three  areas:  the  identification  of  a solid  state  imaging  device  for  cor- 
relation detection,  the  implementation  of  an  electronic  interface  for  automa- 
tic correlation  location,  and  the  development  of  software  for  parallax  collec- 
tion at  30  samples  per  second.  In  addition,  the  potential  of  the  IMF  system 
for  feature  extraction  was  investigated. 

Current  area  imaging  detectors,  including  standard  vidicons,  charge- 
coupled  devices,  and  charge-injection  devices,  were  evaluated  for  possible 
use  as  an  output  correlation  detector.  Based  on  present  performance  levels 
and  a random  access  capability,  the  charge-injection  device  (CID)  was  sel- 
ected. A CIO  camera  system  was  procured  and  installed  in  the  breadboard 
system.  Used  with  phase,  rather  than  amplitude,  matched  filters,  the  CID 
camera  provides  adequate  sensitivity  for  correlation  detection  with  lower 
video  noise  levels  and  a fixed  metric  field. 

An  electronic  interface  providing  the  processing  functions  for  automatic 
correlation  peak  detection  was  designed,  fabricated  and  tested.  Although 
the  electronic  interface  performed  well  in  the  static  test  mode,  several  ef- 
fects limited  the  performance  in  a dynamic  mode.  The  cause  for  the  limited 
performance  was  established,  and  two  possible  corrective  actions  were 
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identified.  The  interface,  while  not  providing  the  potential  accuracy  of 
which  it  is  capable,  did  prove  the  feasibility  of  collecting  parallax  data  at  30 
samples  per  second. 

\ 

Control  software  required  for  system  operation  at  these  rates  was  deve- 
loped. Computer  routines  to  provide  galvanometer  control,  aperture  locat- 
ion and  movement,  and  correlation  peak  detection  were  written  for  the  pro- 
grammable control  unit.  With  the  solid  state  detector  array  in  the  system, 
execution  times  were  consistent  with  data  collection  rates  of  30  samples  per 
second.  In  addition,  data  handling  and  storage  were  provided  at  these  exe- 
cution rates. 

A preliminary  evaluation  of  the  IMF  system  provides  a direct  access  to 
the  input  imagery,  its  Fourier  transform,  and  a spatially  filtered  version  of 
the  imagery.  Although  the  potential  value  of  these  functions  can  be  establi- 
shed only  through  an  in-depth  statistical  study,  initial  estimates  of  their  ef- 
fectiveness were  obtained.  One  example  was  power  spectral  analysis;  the 
energy  distribution  of  power  spectral  density  is  dependent  on  scene  content 
and  structure.  Significant  differences  in  the  power  spectral  density  were 
observed  for  urban  and  rural  regions,  indicating  a relatively  simple  feature 
(such  as  the  power  spectrum)  can  be  effective  for  terrain  classification. 
Additional  discussion  is  provided  in  Section  VI. 


2.  2 CONCLUSIONS  AND  RECOMMENDATIONS 


The  indications  of  the  experimental  effort  to  date  and  the  potential  of 
the  IMF  concept  in  terms  of  cost  and  flexibility  justify  further  system 
investigation.  This  is  particularly  true  with  the  increasing  demand  for  an 
automated  pattern  recognition  scheme. 

Several  areas  for  improvement  have  been  established  during  the  pre- 
sent effort.  To  provide  reliable  operation,  the  scanning  system  should  be 
refurbished.  Although  the  existing  scanning  system  provided  good  perfor- 
mance in  the  past,  these  units  are  due  for.  replacement.  The  existing  open- 
loop  galvanometers  should  be  upgraded  with  closed-loop,  temperature 
regulated  units.  This  will  provide  a significant  increase  in  scanning  accur- 
acy, as  well  as  improving  system  reliability.  This  upgrade  is  warranted 
for  both  stereoccsnpilation  and  feature  extraction. 

The  electronic  interface  designed  and  fabricated  for  correlation  detec- 
tion should  be  modified  to  correct  and  improve  performance  in  the  dynamic 
mode.  Additional  flexibility  should  be  provided  to  use  this  modified  inter- 
face for  both  stereocompilation  and  pattern  recognition  experiments. 

In  addition,  the  possibility  of  using  an  optical  processor  for  pattern 
recognition  from  aerial  photographs  should  be  thoroughly  investigated. 

r 




I 
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Th«  IMF  system  contains  the  basic  elements  required  for  such  an  investi- 
gttiODi  A concentrated,  continuous  study  is  necessary  to  evaluate  the 
effectiveness  of  coherent  optical  processing  for  both  feature  extraction  and 
pattern  recognition. 


SECTION  III 


PRINCIPLES  OF  OPTICAL  DATA  PROCESSING 

Coherent  optica  have  a natural  application  to  the  processing  of  two-dim- 
ensional imagery.  In  contrast  to  digital  computing,  coherent  optics  directly 
provides  parallel  processing  of  large  quantities  of  data  and  is  ideally  mat- 
ched to  sequential  frame  aerial  photography.  Aerial  photographs  contain  a 

g 

tremendous  amount  of  information,  typically  over  10  bits  per  frame.  This 
amount  of  information  is  easily  handled  by  an  optical  system,  but  represents 
a significant  problem  for  digital  techniques  in  terms  of  both  storage  require- 
ments and  processing  speeds.  In  addition,  since  the  optical  system  is  an 
analog  processor,  quantizing  and  digitizing  errors  are  not  encountered.  In 
this  section  we  discuss  the  basic  principles  of  a coherent  optical  processing 
system. 

3. 1 COHERENT  OPTICAL  PROCESSING  TECHNIQUES 

Incoherent  and  coherent  illuminated  optical  systems  have  both  been  used 
for  optical  processing.  The  advent  and  development  of  the  gas  laser  made 
coherently  illuminated  systems  practical  and  superior  in  performance  to  in- 
coherently illuminated  systems.  The  ability  of  a coherent  optical  system  to 
project  a direct  image,  to  display  a Fourier  transform,  and  to  perform  a 
correlation  simultaneously  are  key  advantages. 
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It  is  often  desirable  to  change  the  scale  of  the  transform  in  a coherent 
optical  system.  This  provides  a means  of  varying  the  relative  scale  of  two 
functions  for  optical  filtering.  Various  methods  to  scale  the  transform  exist; 
such  as  a variable  focal  length  lens.  A particularly  simple  method  is  to  em- 
ploy a convergent  beam  geometry  to  perform  the  transform  operation.  This 
geometry  is  utilized  in  the  IMF  system  and  can  readily  provide  a scale 
change  of  + 10%. 

The  basic  elements  of  a coherent  optical  processing  system,  as  imple- 
mented in  the  IMF  system,  are  shown  in  Figure  3-1.  In  describing  the 
operation  of  the  system,  we  assume  that  photographic  transparencies  are 
used  to  input  the  data  for  processing  and  to  record  the  filters  for  correlation 
or  spectrum  weighting.  A point  source  of  monochromatic  light  is  collinated 
by  lens  Lc  and  subsequently  converged  to  plane  Pz  by  the  transform  lens  L0, 
passing  through  a transparency  with  transmittance  t(x).  The  lens  Lz  images 
plane  Pj  into  plane  P3;  in  the  system,  the  relative  positions  of  Pj,  L2,  and 
P3  are  kept  constant  as  the  distance  1 is  varied. 

For  the  one-dimensional  case,  the  light  amplitude  after  passage  through 
the  transparency  is  the  product  of  a converging  wave  and  the  transmittance 
t(x).  This  amplitude,  denoted  as  A(x),  can  be  represented  by 


10 


where 


A(x) 


exp 


A = amplitude  of  illumination  of  L0 
X = wavelength  of  light 
f = focal  length  of  lens  Lq. 


Applying  Kxrchoff'a  formulation  of  Huygens'  principle,  we  see  that  the 
distribution  of  light  in  plane  P is 


T<5) 


where  r is  the  distance  from  a point  x in  plane  P}  to  a point  § in  plane  P2 
and  the  term  (1+cos  6 )/2  is  the  obliquity  factor. 

Because  the  field  angle  6 is  small,  we  make  the  assumption  that  the 

obliguity  factor  (1+cos  6 )/2  is  approximately  one.  Using  the  binomial  ex- 

a xl* 

pansion,  the  distance  r - (t  + ( S ~x)3)  can  be  represented 
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r = 4 


H 


<5  - *P  I (S-JiT 

^ ' 8 l* 


Assuming  -t  is  much  greater  than  § or  x allows  us  to  retain  only  the  first 
two  terms  of  the  expansion;  in  addition,  since  the  first  term  represents  a 
constant  reference  phase,  it  is  dropped  from  the  analysis. 

The  expression  for  T(  § ),  the  amplitude  distribution  in  plane  P^,  then 
becomes 


T(5)  = 


. k x2  . k(g  - x)B 

fA  r t(x)  e 3 21  e 3 21 

j <i  r dx  » 

4 Px  ^ 


where  we  have  observed  that  1/r  varies  slowly  in  comparison  to  the  inter- 

2 

grand.  The  phase  terms  in  x cancel,  allowing  the  simplification 


T(5)  = 


Af 


kg2 

Jir 


-J' 


kff  x 


j t(x)  e 
Pi 


dx. 


The  integral  must  be  performed  over  the  effective  aperture  in  plane  P^;  this 
is  thus  a function  of  t,  the  separation  between  the  transparency  and  the 
transform  plane.  We  require  that  the  half-aperture  W of  lens  L0  be  large 
enough  to  illuminate  the  signal  t(x)  for  all  £ of  interest.  The  effective 
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aperture  size  in  the  transparency  plane  Pj  is  then  simply  Wl/f.  Inserting 
these  limits  of  integration  into  the  expression  for  T(  § ) yields 


T(5)  = g<$,  l)  J*  W‘t’/f  *<*)  « ‘ * <**» 

-Wt/f 

where 

r jMl 

g{*-  l)  =\J\Z  7?  e 

The  expression  for  F(  § ) can  be  arranged  into  a transform  type  relat- 
ion with  the  variable  change  x=  4 £/f  ; this  results  in 

4 w .rp 

T(5)  = — 8 (S,  -t)  J*  t(n*/f)e  du  . 

-W 

Since  t(vi4/f)  vanishes  for  |x|  > W , the  limits  on  the  integral  can  be 
extended  to  infinity.  T(  5)  and  t(M  L /i)  then  form  a Fourier  transform  pair. 
The  phase  factor  g(§  , t)  is  expected  since  the  signal  is  not  in  the  front  focal 
plane  of  LQ.  With  this  system  there  is  no  limit  on  either  the  frequency  res- 
ponse or  the  region  of  space-invariant  operation,  whereas  when  the  signal  is 
placed  in  front  of  LQ,  there  is  a definite  limit  on  both  these  quantities. 
Spherical  aberration  due  to  the  glass  plates  used  to  enclose  the  signal  can  be 


compensated  by  proper  design  of  lens  LQ,  since  it  is  not  a function  of  t. 
However,  we  note  that,  for  the  IMF  system,  the  spherical  aberration  is  not 
a significant  factor.  In  fact,  the  convergent  beam  geometry  generates  less 
aberration  than  the  previous  conventional  set  up  employing  the  custom  trans- 
form lens. 


From  the  transform  relationship  it  is  clear  that  the  effective  scale  of 
the  input  signal  varies  inversely  as  4and  the  size  of  the  transform  varies 
directly  as  This  is  precisely  the  relationship  desired,  since  we  wish  to 
vary  the  size  of  the  transform  with  respect  to  a fixed  scale  in  plane  P2.  A 
spatial  filter  having  an  amplitude  transmittance  S(  § ) can  be  placed  in  plane 
P2  to  modify  the  transform  T(  ? ) directly;  this  capability  is  used  for  perfor- 
ming a correlation  operation.  Alternately  the  Fourier  transform  is  availa- 
ble for  direct  examination  or  spectrum  weighting.  Thu  spectrum  weighting 
functions  available  include  differentiation,  bandpass  filtering,  and  edge-en- 
hancement. 


The  amplitude  r(  u ) of  the  light  in  the  output  plane  P3  of  the  optical  sy- 
stem is  the  Fourier  transform  of  the  wavefront  emerging  from  the  filter. 


i.  e. 


Hu)  = 2“  J*  S(p)  T(p)  e*^pu  dp 
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where  the  spatial  frequency  variable  p is  related  to  the  space  coordinate  § 
in  plane  by 

i 


i 

i 


Here  for  simplicity,  the  focal  lengths  of  lens  L and  L0  are  assumed  to  be 
identical  and  are  denoted  by  f. 


This  relationship  for  r(u)  can  be  used  to  perform  a correlation  opera- 
tion between  two  functions  tj(x)  and  t2(x)  the  filter  function  required  can  be 
generated  as  a spatial  frequency  carrier  filter;  the  filter  can  be  formed  with 
the  same  optical  system  shown  in  Figure  3-1.  The  reference  transparency 
t^(x)  is  placed  in  plane  and  illuminated  by  the  convergent  wavefront  pro- 
duced by  lens  LQ.  The  Fourier  transform  Ti(p)  of  the  transparency  is  dis- 
played in  plane  P2.  An  off-axis  reference  beam  of  collimated  light  is 
incident  on  plane  P2  at  an  angle  a;  this  beam  is  represented  by  R exp  (jpb), 
where  b=  f2  sin  a . The  irradiance  produced  in  plane  P^  i«  denoted  by  S(p) 
and  can  be  represented 


f 


16 


|S(p)!2  = |Rexp(jpb)  + T^p)!2 

= R2  + |T(p)  I2  + Rexp(-jpb)  T^p)  + RexpOpbJTilp) 

This  resultant  interference  pattern  |S(p)  |2  is  recorded  on  a suitable 
recording  medium,  such  as  a high  resolution  photographic  film  or  a photo- 
plastic material.  Materials  of  these  types  typically  exhibit  an  amplitude 
transmittance  Ta  which  is  linear  in  exposure  E;  the  Ta-E  curve  for  the  ma- 
terial is  characterized  as 

T = T - p E, 
a o K 

where  T0  is  the  amplitude  intercept,  $ is  the  slape  of  the  linear  portion  of 
the  Ta-E  curve,  and  E represents  the  exposing  irradiance.  Thus,  after  the 
recording  material  is  exposed  and  processed,  its  amplitude  transmittance 
Ta(p)  is 

Ta(p)  = To  - 3 IS(p)|2 

= 3 [T0/p  - R2  - |S(p)  |2  -p  RTX  (p)  exp(jpb) 

- P RT^pJexp  ( -jpb)  ] 

After  processing,  the  filter  is  replaced  in  the  optical  system  and  is 
illuminated  by  T2(p),  the  Fourier  transform  of  a second  transparency  ti(x). 
In  the  output  plane  P3  of  the  processor,  we  observe  three  filtered  versions 
of  the  input  data? 
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ri  ("u)  = if"  S T«  (P)  L T0/p  - Ra  - I Tx  (p)  I 3 ] e”jpu  dp, 
ra  (-u)  = —j“  J*  Tx  (p)  Ta  (p)e'^p(u'b)  dp, 
rs(-u)  = ; Tx*(p)  Ta  (p)e'jp(u+b)  dp. 

The  first  filtered  wavefront  r (-u)  is  centered  on  the  optical  axis  and  is 
generally  of  little  interest.  The  second  term  r2(-u)  represents  the  convolu- 
tion  of  the  two  transparencies  tj(x)  and  t2(x);  this  term  is  centered  at  u=  +b 
and  thus  separable  from  the  on-axis  wavefront.  The  third  term  r3(-u), 
centered  at  u=  -b,  represents  the  desired  correlation  between  t^x)  and 
t2(x).  The  coordinates  u in  the  output  plane  P3  have  reversed  signs  as  a 
consequence  of  the  image  inversion  caused  by  the  spherical  lenses;  this  in- 
version implies  that  the  filtered  image  is  rotated  through  180  degrees,  a 
natural  result  with  any  imaging  system  whether  incoherently  or  coherently 
illuminated.  The  positive  spherical  lenses  always  introduce  a negative  ker- 
nel function  into  the  Fourier  transform  operation. 

Applying  the  convolution  theorem  to  the  expression  for  r3(-u)  and  noting 
that  the  scale  of  T2(p)  is  dependent  on  the  separation  -t  yields 
r3(-u>  * / tr(x+u)  t2  x ) dx. 

Thus,  the  output  plane  P3  contains  the  distribution  r3(-u),  which  is  the  cor- 
relation integral  of  t^(x)  and  t2(— x).  The  scale  of  the  second  transparency 
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can  be  adjusted  to  match  the  reference  function  tx  (x)  by  varying  the  separa- 
tion t. 

The  capability  for  scale  variation  allows  the  IMF  system  to  use  input 
images  that  may  differ  significantly  in  relative  scales.  Movement  of  the 
image  along  the  optical  axis  would  accomplish  image  scaling.  This  feature 
would  be  of  great  importance  for  the  identification  and  classification  of 
terrain  characteristics. 

3.2  CROSS- CORRELATION  PROCESSING  FOR  PARALLAX  MEASURE- 
MENT 

A coherent  optical  correlator  is  used  to  determine  parallax  by  cross - 
correlating  small  regions  in  a stereo  pair  of  transparencies.  Let  us 
assume  that  one  transparency  h(x#  y)  from  a stereo  pair  is  .used  to  make  a 
spatial  filter  as  described  above.  The  second  transparency  f(x,  y)  and  this 
filter  are  located  respectively  in  planes  Px  and  Pa  of  the  system  shown  in 
Figure  3-1. 

If  only  a small  area  of  the  second  transparency  is  illuminated,  the  out- 
put plane  P*  will  contain  a light  intensity  pattern  corresponding  to  the  cross 
correlation  of  this  illuminated  area  with  that  of  the  entire  first  transparency. 
Ideally,  any  given  small  area  A^  will  cross  correlate  strongly  with  only 
the  corresponding  small  area  Af  of  the  first  transparency.  The  position  of 
the  cross  correlation  peak  in  plane  P«  is  fixed  by  the  relative  positions  of 
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these  two  small  areas  (A^  and  Aj)  in  the  input  gate.  Any  change  in  position 
for  this  region  between  the  filter  making  operation  and  the  cross  correlat- 
ion operation  will  cause  a corresponding  displacement  shift  of  the  correlat- 
ion peak  in  the  output  plane.  This  shift  in  correlation  position  can  be  used 
to  measure  the  parallax  between  the  local  regions.  By  sequentially  illumin- 
ating portions  of  the  second  transparency,  f(x,  y),  and  detecting  the  positions 
of  the  corresponding  correlation  peaks,  we  can  measure  parallax  over  the 
entire  input  scene. 

3.3  OPTICAL  DATA  PROCESSING  FOR  FEATURE  EXTRACTION 

Coherent  optical  processors  in  general  and  the  IMF  breadboard  system 
in  particular  have  considerable  potential  for  feature  extraction.  The  IMF 
system  provides  immediate  access  to  a display  of  the  input  imagery,  the 
Fourier  transform  of  that  imagery,  and  the  parallax  information  for  the 
stereo  pair.  The  simultaneous  presence  of  these  elements  in  a single  sys- 
tem is  a unique  situation  having  significant  potential  for  image  analysis  and 
feature  extraction. 

There  are  three  general  classes  of  operations  which  can  be  performed 
directly  by  a coherent  optical  system.  These  are  power  spectral  analysis, 
simple  spatial  filtering,  and  holographic  spatial  filtering.  Additional  opera- 
tions can  by  implemented  by  combining  the  optical  processor  and  a digital 


computer.  In  this  hybrid  system,  data  samples  provided  by  the  optical  sys- 
tem are  processed  under  programmed  control  by  the  digital  computer!  this 
joint  processing  capability  can  perform  non-linear  operations  not  easily  ac- 
complished with  either  optical  or  digital  systems  separately. 

Power  spectral  analysis  is  simply  accomplished  by  measuring  the  in- 
tensity distribution  in  the  Fourier  transform  plane  of  the  processor.  A 
sensor,  located  in  the  transform  plane  would  sample  the  intensity  under 
computer  direction.  This  process  could  synthesize  any  desired  sampling 
aperture;  discrete,  annular,  or  wedge-shaped  samples  can  be  simulated. 
Significant  changes  in  the  power  spectra  from  various  local  regions  of  the 
input  imagery  can  be  detected;  these  changes  would  outline  the  regions  hav- 
ing certain  textural  characteristics. 

The  introduction  of  various  spatial  filters  into  the  transform  plane  can 
significantly  alter  the  characteristics  of  an  image;  either  simple  or  holo- 
graphic filters  can  be  inserted.  Simple  filters,  such  as  opaque  disks  and 
symmetrical  wedges,  enhance  certain  characteristics  of  the  image.  This 
type  of  image  modification  technique  could  be  used  as  a specialized  viewing 
device  by  a human  interpreter.  Holographic  filters  are  used  to  effect  a 
.cross -correlation  between  the  input  image  and  a reference  scene.  This 
technique  can  be  used  to  identify  and  locate  specific  patterns  as  well  as 
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provide  parallax  information.  However,  the  feature  of  interest  must  be 
quite  similar  in  scale  and  orientation  to  the  reference  pattern  to  produce  a 
significant  correlation  in  the  output  plane. 
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SECTION  IV 


CORRELATION  DETECTION  DEVICES 

During  the  course  of  the  present  contract,  we  investigated  the  possible 
application  to  the  ACME  system  of  three  different  image  sensors:  a stand- 
ard vidicon,  a charge -coupled  device,  and  a charge -injected  device.  Each 
of  the  candidates  utilizes  a different  technique  to  produce  a video  replica  of 
the  intensity  present  in  the  correlation  plane  of  the  processor.  In  this  sec- 
tion, we  first  review  the  basic  requirements  for  the  output  sensor  in  an  au- 
tomatic stereo  compilation  system.  Next,  the  image  sensors  are  discussed 
and  their  relevant  characteristics  are  given.  The  section  is  concluded  with 
a trade-off  analysis  to  establish  the  device  best  suited  for  correlation  detec- 
tion. 

4.1  OUTPUT  SENSOR  REQUIREMENTS 

The  major  function  performed  in  an  automatic  stereo  compilation  sys- 
tem is  the  measurement  of  ground  elevation  profiles.  Typically,  this  meas- 
urement uses  a correlation  technique  which  determines  the  small  parallax 
displacements  of  terrain  regions  in  aerial  photographs.  The  correlation 
technique  employed  can  be  one-  or  two  •dimensional  in  nature.  The  image 
matched  filter  concept  used  in  the  ACME  system  performs  a true  two  - dimen- 
sional cross-correlation  between  the  areas  of  aerial  images.  During  the 
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parallax  collection  operation,  this  cross -cor relation  is  represented  by  the 
irradiance  distribution  in  the  output  plane  of  the  ACME  system.  The  locat- 
ion of  the  maximum  correlation  intensity  in  the  output  plane  provides  a 
measure  of  the  parallax  displacement  and,  hence,  the  local  ground  elevation. 
The  level  of  performance  achieved  by  the  system  is  gauged  by  the  accuracy 
of  the  ground  elevation  values. 

A primary  burden  for  the  recognition  and  definition  of  the  peak  correl- 
ation intensity  is  placed  on  the  two  dimensional  area  sensor  positioned  in  the 
output  plane.  This  output  sensor,  sampling  the  irradiance  distribution  in  the 
output  plane,  provides  the  conversion  from  the  optical  domain  to  the  elec- 
tronic domain.  The  conversion  must  be  consistent  with  overall  system  re- 
quirements. 

Several  system  requirements  influence  the  selection  of  an  output  sensor. 
One  set  of  requirements  is  presented  by  the  photogrammetric  aspects  of  the 
parallax  data.  The  parallax  values  represent  the  small  spatial  displace- 
ments of  terrain  areas  between  two  aerial  scenes.  To  provide  useful  infor- 
mation, the  measured  parallax  values  must  be  referenced  to  a highly  linear, 
metrically  accurate  coordinate  system.  To  obtain  the  degree  of  precision  re- 
quired, any  geometric  distortion  introduced  by  the  output  sensor  must  be 
accounted  for  during  the  data  reduction  process.  The  resolution  required 
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during  the  parallax  measurement  is  also  critical.  For  typical  operational 
parameters,  a difference  in  actual  ground  elevation  of  0.4  m corresponds 
to  a nominal  displacement  in. the  output  plane  (25  urn);  the  output  sen- 
sor must  provide  resolution  on  this  order. 

A separate  set  of  requirements  is  imposed  by  the  optical  detection  as- 
pects of  the  parallax  measurements.  The  intensity  of  the  correlation  peak 
varies  significantly  as  different  areas  of  the  image  are  addressed.  The 
threshold  sensitivity  of  the  output  sensor  must  be  sufficient  to  detect  the  low 
power  signals,  while  the  dynamic  range  of  the  sensor  must  accommodate  the 
intensity  variations.  The  signal-to-noise  ratio  (SNR)  of  the  output  detector 
is  another  consideration.  Any  degradation  of  the  SNR  for  the  optical  correl- 
ation signal  must  be  minimal.  In  addition,  the  SNR  of  the  video  correlation 
signal  must  be  adequate  for  detection  by  subsequent  processing  electronics. 
The  anticipated  data  collection  rate  near  30  frames  per  second  also  impacts 
sensor  choice.  The  frame  rate  fixes  the  maximum  sampling  time  available 
to  the  output  sensor.  The  frame  rate  and  sensitivity  of  the  sensor  jointly 
determine  the  maximum  data  collection  rates. 

Other  properties  of  the  sensor  must  also  be  considered.  The  image 
retention  and  blooming  tendency  of  the  output  sensor  influence  the  accuracy 
of  the  parallax  measurement.  Image  retention  by  the  output  sensor  results 
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from  an  incomplete  erasure  of  previous  images  from  the  sensor  area. 

Image  retention  by  the  sensor  for  an  intense  correlation  peak  would  degrade 
the  system  accuracy  for  subsequent  weaker  correlations.  Similarly,  the 
blooming  tendency  of  the  output  sensor  can  prevent  accurate  correlation 
position  measurements.  Blooming  occurs  at  localized  areas  of  the  sensor 
which  receive  large  overexposures.  The  blooming  causes  the  apparent  size 
of  the  affected  area  to  increase;  this  increased  size  introduces  additional 
inaccuracies  to  the  correlation  measurement  process. 

4.  2 OPERATIONAL  PRINCIPLES 

In  this  section,  we  review  the  operational  principles  of  three  area 
imaging  devices:  the  silicon  vidicon,  a charge  coupled  device  (CCD),  and  a 
charge  injected  device  (CID).  The  silicon  vidicon,  based  on  a well-establis- 
hed technology,  is  routinely  used  for  commercial  applications.  Although 
vidicon  tubes  have  several  desirable  features,  they  do  introduce  geometric 
distortion  into  the  scanning  process;  this  distortion  is  a key  consideration 
for  a parallax  measurement  application.  The  two  solid  state  arrays,  the 
CCD  and  CID  area  imagers,  offer  excellent  metricity.  However,  these 
devices  do  not  presently  achieve  the  resolution  obtained  by  electron  beam 
tubes.  To  establish  the  limiting  performance  of  the  three  imaging  devices, 
their  basic  contraction  and  operation  must  be  considered. 

i 

A. 
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The  standard  silicon  vidicon  is  an  electron  tube  device  which  converts 


an  optical  image  into  a video  signal.  The  basic  construction  and  operation 
of  a vidicon  relies  on  a scanning  electron  beam  and  a photoconductive  tar- 
get. A transparent  conductive  layer,  coated  on  the  front  surface  of  a photo- 
conductor, acts  as  a signal  electrode.  During  operation,  the  photoconductor 

t 

is  initially  charged  by  a scanning  electron  beam  to  a uniform  surface  vol- 
tage which  creates  an  electric  field  within  the  photoconductive  layer.  An 
optical  image  focused  on  the  photoconductor  surface  generates  charge 
carriers  which  migrate  under  the  influence  of  the  applied  field  to  the  photo- 
conductor surfaces.  These  migrating  charges  modify  the  original  surface 
voltage  present,  forming  an  electrostatic  latent  image  of  the  optical  image. 
The  electron  beam,  whose  size  and  shape  are  controlled  by  focus  and  de- 
flection coils  surrounding  the  tube,  scans  in  a raster  type  fashion  across 
the  photo  con  due  tor  and  deposits  a surface  charge  in  proportion  to  the  local 
surface  voltage.  The  charge  deposition  process  generates  a capacitve  dis- 
placement current  at  the  signal  electrode  and  returns  the  surface  voltage  to 
its  original  uniform  value.  The  capacitive  displacement  current,  related  to 
the  original  optical  energy  incident  on  the  tube,  is  used  to  drive  the  display 
and  processing  electronics. 

The  charge-coupled  imaging  device  (CCD)  represents  one  technique 
that  achieves  the  capability  of  complete  solid-state  imaging.  The  CCD 
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imager  is  composed  of  photosensor  gates,  a vertical  line  analog  shift  regis- 
ter, and  a horizontal  line  shift  register.  The  operation  of  the  device  is  sub- 
divided into  three  phases:  integrate,  transfer,  and  shift.  These  phases  are 
initiated  by  the  clocked  voltage  levels  present  on  the  photosensor  gates, 
scan  gates,  and  shift  registers.  During  the  integrate  period,  each  photo- 
sensor of  the  array  is  allowed  to  accumulate  a charge  packet  proportional 
to  the  local  light  energy.  A low  voltage  level  on  the  photosensor  gate  local- 
izes the  charge  packet  beneath  the  gate  and  prevents  any  charge  transfer. 
During  the  vertical  blanking  time,  the  photogenerated  charge  packet  is  tran- 
sferred into  the  adjacent  CCD  shift  register  element.  The  charge  transfer 
is  enabled  by  a simultaneous  high  voltage  level  on  the  photosensor  gate  and 
a low  voltage  level  on  the  scan  gate.  The  potential  barriers  generated  by 
these  voltage  levels  transfer  the  charge  packets  into  the  adjacent  elements 
of  the  vertical  shift  register.  During  the  shift  phase,  each  row  of  the 
charge  array  stored  in  the  vertical  register  is  transferred  into  the  horizon- 
tal shift  register;  during  this  shift  phase,  the  photosites  are  again  allowed 
to  accumulate  charge.  The  horizontal  shift  register  is  sequentially  clocked 
to  produce  the  video  replica  of  the  original  image. 


The  charge-injected  imaging  device  (C1D)  represents  an  alternative 
technology  for  solid  state  imaging  systems.  The  basic  operation  of  the  CID 
is  similar  to  that  of  the  CCD.  The  CID  operation  is  also  divided  into  three 
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phases;  integrate,  transfer,  and  inject.  As  in  the  CCD  technology,  these 
phases  are  initiated  by  clocked  voltage  levels  on  the  photosensor  and  trans- 
fer gates.  During  the  integrate  phase  the  photosensor  sites  are  allowed  to 
accumulate  a charge  packet  corresponding  to  the  incident  light  intensity. 
When  one  row  of  the  array  is  selected  for  readout,  the  charge  packet  is 
transferred  to  the  adjacent  charge  storage  elements.  The  transfer  is  initi- 
ated for  a particular  photosite  by  a simultaneous  high  voltage  level  on  the 
photogate  and  low  voltage  level  on  the  transfer  gate.  A selection  circuit  for 
the  CID  sequentially  transfer  the  charge  packets  from  the  photosensor  area 
to  the  storage  areas.  Although  normally  operated  in  this  sequential  mode, 
the  CID  does  possess  the  capability  for  random  access  addressing.  The 
charge  packet  stored  at  each  site  is  sensed  by  injection  into  the  bulk  silicon 
substrate  and  detection  of  the  associated  displacement  current.  An  on-chip 
amplifier  converts  the  displacement  current  into  a video  replica  of  the 
original  image. 

4.  3 DEVICE  TRADE  - OFF  ANALYSIS  AND  SELECTION 

The  relative  performance  levels  achieved  by  the  three  types  of  imaging 
devices  are  summarized  in  Table  4-1.  The  devices  summarized  in  the 
table  are  commercially  available  and  are  generally  representative  of  their 
respective  technologies.  The  vidicon,  an  MTI  Type  V-440,  provides  a 
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scanning  rate  for  625  TV  lines  of  30  frames  per  second.  The  CCD  imaging 
device,  a Fairchild  CCD-211,  provides  scanning  of  488  video  lines  at  rates 
up  to  100  frames  per  second.  The  CID  imaging  device,  incorporated  in  the 
General  Electric  Z7892  CID  television  camera,  provides  a resolution  of 
488  video  lines  at  a 30  frames  per  second  scan  rate.  Both  solid  state  de- 
vices utilize  a multiplexing  technique  to  achieve  their,  full  resolution. 

The  non-linearity  and  geometric  distortion  are  significantly  higher  for 
the  vidicon  than  for  the  solid  state  sensors.  The  predominant  cause  for  this 
effect  is  the  deflection  system  employed  in  the  vidicon.  Typically,  pincush- 
ion distortion  is  the  main  aberration  causing  the  nonlinearity.  The  use  of  a 
parabolic  sweep  generator  and  precision  deflection  yokes  for  the  vidicon  can 
reduce  the  pincushion  distortion  significantly;  however,  the  high  scan  fidelity 
is  attained  only  with  the  sacrifice  of  deflection  speed  and  an  increase  in  de- 
vice cost.  The  linearity  and  distortion  levels  achieved  for  the  solid  state 
sensors  is  completely  dependent  on  fabrication  techniques.  In  addition,  the 
presence  of  extraneous  magnetic  fields  does  not  compromise  the  perfor- 
mance levels  for  the  solid  state  sensors. 

The  resolution  achieved  by  vidicons  is  higher  than  that  of  current  solid 
state  sensors.  The  resolution  for  the  vidicon  is  limited  by  the  bandwidth  of 
the  focussing  coil  and  camera  chain.  The  size  of  the  scanning  electron 

beam  is  fixed  by  the  focussing  coil.  Whereas  the  time  response  is  deter- 

\ 

mined  by  bandwidth  constraints.  Present  state-of-the-art  vidicons  are 
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limited  to  a maximum  resolution  of  2000  lines  at  TV  frame  rates.  Present- 
ly, solid  state  sensors  do  not  achieve  this  resolution.  Typical  resolutions 
achieved  with  the  solid  state  devices  are  half  that  of  standard  vidicons.  How- 
ever, the  resolution  of  available  CCD  and  CID  arrays  will  improve  as  fab- 
rication experience  and  techniques  are  developed. 


The  sensitivity,  signal-to-noise-  ratio,  and  dynamic  range  are  also  im- 
portant system  considerations.  These  parameters  establish  the  operating 
characteristics  of  the  image  sensor.  The  sensitivities  listed  in  Table  4-1 
are  for  particular  devices  and  are  not  indicative  of  fundamental  technology 

limitations;  sensors  are  available  which  function  in  ambient  light  levels  of 
-6 

10  footcandles.  The  vidicon  and  CID  presently  have  sensitivities  in  the 
range  necessary  for  the  ACME  system.  The  higher  sensitivity  provided  by 
the  CCD  imager  would  necessitate  added  attenuation  in  the  optical  system. 
Precautions  to  ensure  no  stray  light  degradation  would  also  be  necessary. 
The  signal-to-noise  ratio  and  dynamic  range  provided  by  the  sensors  are 
consistent  with  correlation  recognition  and  detection. 

Image  retention  or  lag  by  the  output  sensor  is  another  practical  consid- 
eration. Image  lag  by  the  output  sensor  allows  a signal,  although 
attenuated,  to  persist  for  several  video  frames  after  its  initial  occurence. 
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The  image  lag  properties  of  the  solid  state  devices  are  superior  to  those 
of  the  vidicon.  Image  retention  of  the  vidicon  results  from  an  incomplete 
erasure  of  the  photo-induced  surface  voltage.  The  amount  of  retention  is 
dependent  on  signal  level;  the  table  listing  is  representative  of  the  retention 
for  an  average  signal  level.  Image  retention  in  the  solid  state  devices  is 
caused  by  an  incomplete  transfer  of  charge  from  the  photosites.  A typical 
level  of  the  transfer  inefficiency  averaged  over  a video  frame  is  0.  5 x 10"^. 
No  degradation  is  caused  by  transfer  inefficiency  for  solid  state  arrays  with 
element  densities  in  the  range  anticipated. 

The  high  metric  fidelity  required  for  the  stereo  compilation  system  is 
presently  achieved  only  by  the  solid  state  imaging  arrays.  To  achieve  the 
same  level  with  a vidicon  would  require  a substantial  increase  in  both  devic. 
complexity  and  expense.  Similar  limitations  exist  for  increasing  the  frame 
rate  capability  of  the  vidicon;  special  yoke  and  electronic  design  are  requir 
ed  to  achieve  faster  frame  rates.  These  limitations  eliminate  the  vidicon 
as  a viable  candidate  for  a stereo  compilation  system. 

The  two  remaining  candidates,  the  CCD  and  CID  area  imagers,  have 
comparable  performance.  Of  these  two,  the  CID  area  imager  is  easier  to 
fabricate.  This  typically  yields  CID  imagers  with  fewer  blemishes  or  bar 
sites  than  comparable  CCD  devices.  The  readout  process  for  the  CID  is 
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• lightly  noisier  than  that  for  the  CCD;  the  increased  noise  arises  from  the 
higher  output  capacitance  of  the  CID  area  imager.  However,  an  acceptable 
SNR  can  be  achieved  with  both  devices.  The  readout  stream  from  the  CCD 
is  necessarily  serial  in  nature;  the  CID  can  be  addressed  in  an  x-y  matrix 
fashion.  Thus,  the  CID  has  a capability  for  random  access  readout.  The 
random  access  feature  is  desirable  for  an  application  such  as  stereo  com- 
pilation. The  random  access  ability  of  the  CID  supports  fast  readout  rates 
while  still  providing  metric  accuracy.  As  the  technology  for  charge  inject- 
ion devices  matures,  it  is  reasonable  to  expect  that  even  higher  resolution 
(over  1000  elements  per  axis)  can  be  achieved.  The  CID  area  imager,  with 
its  random  access  capability,  is  the  preferred  detector  for  an  engineering 
model  stereo  compilation  system. 


SECTION  V 


IMF  SYSTEM  CONFIGURATION 

A major  goal  of  this  contract  was  to  develop  and  refine  techniques  for 
the  automatic  extraction  of  parallax  data  from  aerial  photographs.  In  parti- 
cular, a hybrid  electro-optical  system  capable  of  collecting  parallax  data  at 
rates  of  30  samples  per  second  was  implemented.  In  this  section  of  the  re- 
port we  present  a functional  description  of  this  system  concept.  The 
description  details  the  optical  and  electronic  subsystems  involved  and 
covers  the  control  functions  provided  by  software.  The  section  is  concluded 
by  a discussion  of  the  development,  results  and  status  of  the  present  experi- 
mental system. 

5.  1 IMF  SYSTEM  DESCRIPTION 

Cross  correlation  techniques  are  used  in  the  Image  Matched  Filter 
(IMF)  system  to  measure  terrain  profile.  The  cross-correlation  operation, 
giving  the  best  Linear  estimate  of  similarity,  is  dependent  on  the  relative 
displacements  between  the  two  functions  evaluated.  Thus,  the  position  of 
the  cross-correlation  peak  gives  a direct  measure  of  the  relative  displace- 
ment, or  parallax. 

A key  end  use  identified  for  the  IMF  system  is  the  fast  and  accurate 
measurement  of  this  parallax.  The  measurement  process  involves  deter- 
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mining  small,  typically  50  to  100  um,  displacements  in  the  relative 
position  of  a terrain  region  in  two  separate  aerial  photographs.  The  paral- 
lax measurement  must  be  performed  at  0.  5 mm  intervals  over  the  photo- 
graphs forming  the  stereo  pair. 

Several  key  subsystems  are  required  in  the  IMF  system  for  the  paral- 
lax measurement  process.  An  optical  subsystem  must  be  assembled  to 
perform  a cross-correlation  between  two  functions  contained  in  transparen- 
cy form;  a Fourier  transform  geometry  is  employed  to  cross-correlate  one 
entire  scene,  stored  as  an  image  matched  filter,  and  a small  localized  re- 
gion of  the  second  scene.  An  electronic  subsystem  must  provide  the  neces- 
sary functions  to  operate  the  system;  these  functions  include  accessing  a 
specified  location  in  the  second  scene  and  determining  the  position  and 
intensity  of  the  correlation  peak.  The  overall  timing  and  control  for  opera- 
tion and  the  actual  data  collection  and  storage  must  be  accomplished  by  a 

$ 

programmable  control  unit  which  monitors  the  process. 

The  interrelationships  and  functions  of  these  subsystems  are  shown  by 
the  block  diagram  in  Figure  5-1.  Under  software  control  the  programmable 
control  unit  (PCU)  selects  a specified  location  in  the  second  transparency 
at  which  parallax  is  to  be  measured.  Binary  values  corresponding  to  this 
physical  location  are  routed  to  digital-to-analog  converters  (D/A);  the 


D/A  unit*  convert  the  binary  signal  into  an  analog  voltage  for  the  galvano- 
meter drivers  which  then  steer  the  scanning  beam  to  the  desired  location. 
The  optical  subsystem  performs  the  cross-correlation  between  this  location 
and  the  entire  first  scene  of  the  stereo  pair.  This  cross -correlation  is 
detected  by  a two  dimensional  area  sensor,  which  in  the  present  configura- 
tion is  a charge-injection  device. 

. The  effective  field  of  view  of  the  sensor  is  limited  by  a video  gating 
circuit.  Although  the  correlation  peak  is  not  always  the  brightest  point  in 
the  entire  output  plane,  the  peak  is  usually  the  brightest  in  a local  region. 
Thus,  by  limiting  the  effective  field,  the  peak  can  be  tracked  through  areas 
of  weak  correlation.  The  video  gating  circuit  limits  the  field  of  view  to  a 
rectangular  area  of  the  sensor;  the  size  and  location  of  the  aperture  are 
selectable  by  software  commands  from  the  PCU.  The  gating  circuit  allows 
only  the  video  signal  from  a specified  region  to  reach  the  correlation  peak 
detector  module.  The  maximum  video  peak  within  this  specified  region  is 
detected  as  the  true  correlation,  and  its  location  and  intensity  are  stored 
for  any  subsequent  processing  or  data  reduction. 

At  this  point  the  parallax  values  for  one  specified  location  have  been 
established  and  stored.  Depending  on  the  stage  of  program  completion,  a - 
nother  scan  point  can  be  selected  for  parallax  measurement  or  a data  re- 


38 


duction  and  display  routine  could  be  called  for  execution. 

5.  1.  1 IMF  OPTICAL  SUBSYSTEM 

The  IMF  optical  subsystem  used  in  this  study  has  two  functional  modes: 
the  filter  generation  mode  and  the  data  collection  mode.  The  present  config- 
uration of  the  system  for  filter  generation  is  shown  in  Figure  5.  1.  1-1,  and 
that  for  data  collection  is  shown  in  Figure  5.  1.  1-2,  The  components 
necessary  for  both  modes  of  operation  are  present  simultaneously  on  the 
pneumatically  supported  optics  table;  the  interchange  between  modes  is 
easily  accomplished  by  repositioning  only  two  mirrors:  the  beam  routing 
mirror  Mj  and  the  scanning  mode  mirror  M2.  A lens -pinhole  spatial  filter 
assembly  determines  the  correct  orientation  of  Mj,  and  fixed  mechanical 
stops  locate  M2* 

In  the  filter  generation  mode  (Figure  5.  1.  1-1),  the  514.  5 nm  light  from 
a Spectra  Physics  165  Argon  ion  laser  is  directed  into  the  system  by  beam 
routing  mirror  Mj.  The  incident  laser  beam  is  split  into  a signal  and  a 
reference  beam  by  a beamsplitter.  The  reference  beam  is  then  spatially 
filtered  and  collimated  with  its  axis  at  an  angle  of  30°  to  the  signal  beam 
axis. 

The  signal  beam  forms  a point  source  at  P which  is  imaged  into  fil- 
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BS  - BEAM  SPLITTER  G - LIQUID  GATE 

A - ADJUSTABLE  ATTENUATOR  Lc  - COLLIMATING  LENS 

L0  - MICROSCOPE  OBJECTIVE  P -PINHOLE 

Lt  - FOURIER  TRANSFORM  LENS 

Figure  5.  1.  1-1  IMF  Optical  Subsystem;  Filter  Generation  Mode 


Ml  - BEAM  ROUTING  MIRROR  S - GALVANOMETER  SCANNING  UNIT 

L0  - MICROSCOPE  OBJECTIVE  M2  - SCANNING  MODE  MIRROR 

P - PINHOLE  Lc  - COLLIMATING  LENS 

M -MIRROR  G -LIQUID  GATE 

Lj  - FOURIER  TRANSFORM  LENS  L - CORRELATION  LENS 


D - CORRELATION  DETECTOR 


Figure  5.  1.  1-2  IMF  Optical  Subsystem  - Data  Collection  Mode 


41 


ter  plane  by  means  of  the  lens  combination  Lc  and  Lj.  The  input  transpar- 
ency is  held  in  a liquid  gate  G in  the  convergent  region  of  the  signal  beam. 
This  transform  has  a scale  determined  by  the  distance  between  the  input  and 
the  focus  of  the  signal  beam.  This  dependence  upon  input  to  frequency  plane 
separation  can  be  used  to  correct  for  scale  differences  between  input  trans- 
parencies. However,  such  scale  correction  was  not  employed  in  any  of  the 
work  reported  here. 

The  image  matched  filter  for  the  input  is  produced  by  positioning  a 
high  resolution  photographic  plate  in  the  Fourier  plane  P£  and  exposing 
the  plate  to  the  reference  and  signal  beams  simultaneously.  The  relative 
intensities  (K- ratio)  of  the  two  beams  are  measured  with  an  annular  fiber 
optic  array  coupled  to  a photodetector.  The  array  accepts  energy  in  a band 
from  2 cy/mm  to  5 cy /mm.  After  exposure  and  processing,  the  photo- 
graphic plate  containing  the  image  matched  filter  is  repositioned  in  the 
optical  Fourier  plane  P2»  The  system  is  now  ready  for  parallax  data  collec- 
tion. 


The  optical  system  arrangement  for  data  collection  is  shown  in  Figure 
5.  1.  1-2.  To  change  to  the  data  collection  mode,  the  beam  routing  mirror 
Mi  is  reoriented  so  that  all  of  the  laser  light  is  directed  to  a telecentric 
galvanometer  scanner.  The  beam  in  this  case  is  formed  in  a narrow  cone 
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which  illuminates  an  area  approximately  1 mm  in  diameter  at  the  input  gate. 
The  scanning  beam  routing  mirror  M2  is  translated  into  position  against 
fixed  stops  when  converting  to  the  data  collection  mode. 


The  telecentric  scanner  consists  of  two  small  galvanometers  whose 
axes  are  at  right  angles  and  whose  mirrors  are  in  the  conjugate  planes  of  a 
folded  telescope.  The  scanner  is  positioned  so  that  the  scanning  beam  pin- 
hole is  imaged  onto  the  center  of  both  mirrors.  This  assures  that  the 
scanning  beam  always  appears  to  come  from  the  same  point  regardless  of 
the  direction  in  which  it  is  deflected.  The  scanner  and  scanning  beam 
optics  are  arranged  so  that  the  scanning  beam  pinhole  and  the  telecentric 
scanner  are  optically  at  the  same  position  with  respect  to  the  lens  pair  Lc 
and  Lt  as  the  signal  beam  pinhole  was  during  filter  generation.  When  the 
system  is  properly  aligned,  the  image  of  the  scanning  beam  pinhole  remains 
stationary  regardless  of  the  position  of  the  scanning  beam  in  the  input  gate. 


The  image  matched  filter,  when  illuminated  by  a wavefront  derived 
from  a corresponding  area  of  the  input  transparency,  reconstructs  a modi- 
fied version  of  the  original  reference  beam  at  a nominal  angle  of  30°  to  the 
optical  axia.  This  beam  is  reflected  by  the  mirror  to  the  lens  L which 


forms  the  cross  correlations  in  the  output  (correlation)  plane.  This  lens 
and  mirror  are  mounted  on  a small  auxiliary  optical  rail  attached  to  the  top 
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of  the  optical  table. 


The  cross-correlation  is  detected  by  a two-dimensional  area  sensor, 
which  in  the  present  configuration  is  a charge  injection  imaging  device,  the 
The  camera  converts  the  optical  signal  into  an  electronic  video  signal  for 
subsequent  processing  and  correlation  detection. 

Additionally,  a two  dimensional  fiber  optic  array  is  mounted  on  the  in- 
put gate.  This  array  consists  of  « horizontal  and  v&rtica*  row  of  fibers  posi- 
tioned along  two  adjacent  edges  of  the  input  gate.  These  fibers  lead  to  a 
single  photodetector  whose  output  is  sensed  by  an  A/D  converter  interfaced 
to  the  programmable  control  unit.  By  moving  the  scanning  beam  along 
this  array,  the  PCU  can  automatically  calibrate  and  linearize  the  galvano- 
meter scanner. 

5.  1.  2.  IMF  ELECTRONIC  SUBSYSTEM 

The  automatic  recognition  and  detection  of  the  correlation  peak  invol- 
ves a number  of  operations.  First,  the  correct  correlation  peak  must  be 
identified.  Although  this  operation  is  relatively  straightforward  for  regions 
of  high  signal-to-noise  ratio  (SNR)  and  adequate  intensity,  it  is  generally 
not  possible  on  a global  basis.  The  correlation  peak  typically  has  a high 
localized  SNR,  but  is  not  the  most  intense  point  in  the  entire  output  plane. 
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Thus,  the  effective  field  of  view  for  the  sensor  must  be  limited  in  regions 
of  weak  correlation.  In  addition,  the  position  of  this  limited  field  of  view 
must  be  controlled  by  software  commands  to  tract  the  changing  correlation 
function.  Second,  the  correlation  peak  location  must  be  measured  with  high 
precision.  This  measurement  process  must  be  consistent  with  high  speed 
processing,  while  not  requiring  excessive  bandwidths.  Third,  the  technique 
should  be  compatible  with  several  output  detector  types,  including  standard 
vidicon  imaging  tubes  and  solid  state  arrays. 

Other  functions  must  be  provided  by  the  electronic  subsystem.  The 
interface  must  provide  the  drive  signals  for  scanner  operation;  the  drive 
signals  should  be  compatible  with  software  control  to  correct  for  changes 
such  as  scene  orientation  or  scale  drifts.  The  electronics  must  also  meas- 
ure the  intensity  of  the  correlation  peak.  The  intensity  is  useful  in  establish- 
ing the  field  of  view  for  the  sensor.  In  addition,  this  feature  can  provide 
a means  to  sample  and  quantize  an  arbitrary  intensity  distribution;  this 
capability  could  be  applied  to  power  spectral  analysis. 

We  developed  a correlation  peak  tracking  technique  using  these  above 
considerations  as  design  goals.  The  technique  is  based  on  the  use  of  a peak 
stretching  circuit  in  conjuction  with  a clock  and  counter.  The  peak  stretch- 
ing circuit  determines  the  most  intense  point  contained  in  the  video  signal; 
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the  clock  and  counter  define  the  location  of  this  point.  A video  gating 
circuit  is  added  to  the  peak  detection  module  to  allow  only  a selected  area 
of  the  output  plane  to  be  processed. 

The  functional  schematic  of  the  IMF  electronic  subsystem  is  shown  in 
Figure  5.  1.  2-1.  The  programmable  control  unit  (PCU),  a Digital  Equip- 
ment Corporation  PDP  - 8A  minicomputer,  provides  the  central  direction 
for  the  parallax  data  collection  process.  Under  software  control  the  PCU 
establishes  the  boundaries  of  the  rectangular  region  of  the  output  plane 
selected  for  processing.  These  boundaries  are  defined  by  four  digital 
words  which  designate  the  TV  line  numbers  and  times  along  these  lines 
which  limit  the  field  of  view.  These  words  are  loaded  into  the  buffers  of 
the  aperture  location  module  and  are  retained  until  updated.  These  buffers 
control  the  start  and  stop  action  of  a gated  amplifier  in  the  sync  removal 
and  video  gating  circuit.  The  input  to  the  amplifier  is  the  composite  video 
signal  from  the  camera  which  is  located  in  the  correlation  plane.  The  hori- 
zontal and  vertical  sync  signals  from  the  camera  drive  a frame  synchronous 
clock  which  runs  at  200  times  the  TV  line  rate;  this  clock  is  used  to  meas- 
ure time  along  a scan  line.  Digital  comparitors  in  the  video  gating  circuit 
monitor  the  values  of  the  frame  synchronous  clock  and  TV  line  number. 
When  these  valves  are  within  the  range  specified  by  the  aperture  location 
module,  the  composite  video  signal  is  gated  through  to  the  peak  detector. 
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Figure  5.  1.  2-1  Functional  Schematic  of  the  IMF  Electronic  Subsystem 


For  values  outside  this  range,  the  composite  video  is  blocked.  Thus,  only 
the  region  defined  by  the  PCU  is  searched  for  the  correlation  peak. 

The  peak  detector  examines  the  video  signal  for  the  most  intense 
point  with  a peak  stretching  circuit  consisting  of  a diode  and  capacitor 
arranged  to  form  a one-directional  charging  circuit.  As  the  incoming  video 
signal  increases,  the  capacitor  is  allowed  to  charge  and  to  follow  the  input. 
However,  when  the  video  signal  decreases,  the  capacitor  is  prevented  from 
discharging  by  the  forward-biased  diode.  If  the  incoming  video  signal  sub- 
sequently exceeds  the  previous  maximum,  the  capacitor  again  charges. 

As  the  capacitor  charges,  the  timer,  providing  the  x-position  of  the  peak, 
is  allowed  to  count  the  values  from  the  frame  synchronous  clock.  When  the 
capacitor  stops  charging  the  count  of  the  timer  is  frozen  and  indicates  the 
position  of  the  peak.  This  circuit  could  stretch  the  video  peak  over  an  en- 
tire TV  frame;  however,  to  ease  the  requirements  on  the  droop  character- 
istics of  the  circuit,  the  peak  detector  operates  only  on  a line-by-line  basis. 
A reset  pulse  at  the  end  of  each  raster  line  shorts  the  capacitor  and  reiniti- 
alizes the  circuit. 


Two  track/hold  modules  follow  the  peak  detector.  These  modules  have 
different  time  constants  to  provide  peak  acquisition  over  the  TV  frame  per- 
iod. The  first  track/hold  module,  operating  over  a short  time  frame. 
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acquires  the  peak  valves  for  contiguous  lines  and  compares  them  to  deter- 
mine the  maximum  valve.  Each  time  a maximum  peak  is  encountered,  the 
line  counter  is  loaded  with  the  corresponding  line  number.  At  the  conclu- 
sion of  a frame  the  timer  and  line  counter  contain  the  valves  corresponding 
to  the  peak  location. 


The  second  track  /hold  module  follows  the  peak  valve  over  the  time  in- 
terval specified  by  the  contnets  of  the  aperture  location  buffers.  At  the  con- 
clusion of  the  frame,  the  peak  intensity  held  by  the  second  track/hold  mod- 
ule is  converted  into  a digital  word  and  loaded  into  a buffer.  The  PCU  by 
polling  the  timer,  line  counter,  and  buffer  can  establish  the  x-position, 
y -position,  and  intensity  of  the  correlation  peak.  The  PCU  stores  these 
values  and  selects  the  next  scan  location.  The  aperture  location  is  updated 
and  the  process  is  repeated  for  the  new  scan  location. 


The  schematics  for  the  circuits  implemented  for  the  IMF  system  are 
presented  in  Appendix  A.  The  appendix  details  the  computer  interface  cir- 
cuits, the  aperture  location  module,  and  the  track/hold  circuits.  In  addit- 
ion, the  analog  electronic  circuit  which  performs  the  peak  detection  is 
included. 

The  analog  detection  module  was  first  fabricated  using  a vector  board 
construction*  Initial  testing  of  this  module  revealed  a noise  problem  caused 
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by  low  signal  levels  and  incomplete  component  isolation.  To  remedy  this 
situation,  the  analog  detection  module  was  subsequently  fabricated  on  a cop- 
per-clad printed  circuit  board  which  provided  a positive  ground  plane  and 
increased  noise  isolation.  Preliminary  tests  of  this  module  showed  impro- 
ved noise  performance  in  a static  testing  mode.  The  digital  interface  cir- 
cuits were  fabricated  on  standard  Digital  Equipment  Corporation  cards  and 
installed  in  the  programmable  control  unit.  Complete  details  of  the  inter- 
face performance  are  given  in  Section  5.2  of  this  report. 

5.  1.  3 IMF  SOFTWARE  CONTROL  FUNCTIONS 

The  control  functions  for  the  IMF  system  are  provided  by  the  program- 
mable control  unit  through  software  routines.  The  programmable  control 
unit  is  a Digital  Equipment  Corporation  PDP-8A  minicomputer  with  32k 
words  of  on-line  starage  capacity  and  dual  floppy  disc  drive  units  for  off- 
line storage.  The  major  controlling  programs  are  written  in  FORTRAN-II, 
a high  level  language  chosen  for  its  rapid  execution  times.  Support 
programs  and  subroutines  utilize  SABR®’  , a language  commonly  used  on 
the  PDP  family  of  minicomputers  from  Digital  Equipment  Corporation. 

These  two  languages  were  selected  for  use  because  of  their  relative  pro- 
gramming simplicity.  Device  handlers  providing  control  over  external 
equipment  can  be  developed  in  the  SABR  language;  the  SABR  handlers  can 
subsequently  be  used  directly  by  the  main  FORTRAN-II  calling  program. 
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Interface  options  are  available  on  the  IMF  system  to  support  computer 
control  of  all  major  operations  including  the  linearization  and  calibration  of 
the  galvanometer  scanner,  the  drive  and  control  of  the  galvanometer  scan- 
ner, and  the  automatic  collection  of  parallax  data  for  stereocompilation. 
During  previous  contractual  phases,  the  scanner  linearization  feature  has 
been  demonstrated.  During  the  present  effort,  this  capability  was  not  em- 
ployed; however,  the  hardware  components  and  interface  ports  are  available 
in  the  present  system. 

The  scanner  linearization  can  be  accomplished  by  utilizing  the  fiber 
optic  array  located  at  the  periphery  of  the  input  liquid  gate.  The  L-shaped 
array  consists  of  250  u m diameter  optical  fibers  spaced  on  5.00  mm 
centers;  the  fiber  array,  terminated  at  a common  photodetector,  does  not 
block  the  active  scan  region  of  the  gate.  To  linearize  the  scanner,  the 
operator  initially  directs  the  scanning  beam  to  the  corner  fiber  of  the  array. 
Under  programmed  control  the  beam  is  incrementally  stepped  across  the 
horizontal  and  vertical  legs  of  the  array.  At  each  position  of  the  beam,  an 
A/D  conversion  of  the  photodetector  output  can  be  performed.  By  locating 
the  positions  that  correspond  to  maximum  transmitted  power  levels,  the 
computer  can  develop  an  array  of  digital  valves  corresponding  to  known 
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physical  locations.  This  data  can  be  stored  and  used  to  linearize  the  scan- 
ning system.  In  order  to  calibrate  these  parameters  in  terms  of  direct 
photocoordinates,  the  operator  must  direct  the  scanning  beam  to  several 
reseaus  in  the  input  scene.  Software  developed  during  previous  contractual 
phases  can  perform  the  necessary  transformation  from  the  fiber  optic  array 
coordinates  to  the  input  photocoordinates;  this  transformation  accounts  for 
scale  changes  as  well  as  rotations. 

The  control  of  the  scanning  system  is  provided  by  two  digital -to -analog 

* 

(D/A)  converters  in  the  computer  interface.  Binary  valves  corresponding  to 
given  physical  locations  are  sent  to  the  D/A  converters;  these  valves  are 
subsequently  converted  to  analog  levels  which  are  supplied  to  the  galvano- 
meter drivers.  Computer  controlled  operation  of  the  scanning  system  was 
demonstrated  on  earlier  program  phases  where  its  accuracy  and  valve  was 
proven.  A separate  interface  compatible  with  the  programmable  control 
unit  and  different  software  device  handlers  consistent  with  the  new  operating 
system  weru  developed  to  utilize  the  automatic  scanning  process. 

The  package  for  automatic  parallax  collection  was  similarly  developed; 
this  package  contained  the  electronics  required  for  video  gating  and  peak 
detection,  as  well  as  the  circuitry  for  reading  the  correlation  position  and 
intensity.  The  software  required  to  provide  the  timing  and  control  of  the 
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interface  electronics  was  also  developed  for  the  new  PCU.  This  software 
package  also  provides  the  internal  storage  and  a video  storage  of  the  collec- 
ted data. 


The  flowchart  indicating  the  operation  of  the  data  collection  program  is 
shown  in  Figure  5.1.  3;  a complete  program  listing  is  contained  in  Appendix 
B.  Correct  operation  requires  that  the  scanning  sequence  is  initialized. 

This  off-line,  manual  operation  involves  the  adjustment  of  input  scene  orien- 
tation and  the  selection  of  Nl,  the  number  of  data  samples  per  parallax  point. 
The  scene  orientation  is  checked  by  comparing  the  projected  image  of  the 
input  scene  with  the  reconstructed  image  of  the  reference  scene.  The  num- 
ber of  data  samples  per  parallax  point  is  specified  in  response  to  computer 
prompting  and  represents  the  number  of  consecutive -video  frames  over  which 
the  correlation  peak  is  averaged. 


After  the  scanning  sequence  has  been  initialized,  the  computer  directs 
the  scanning  beam  to  the  first  point  at  which  parallax  data  is  to  be  collected. 
The  correlation  peak,  indicating  the  parallax  valves  for  this  initial  point,  is 
displayed  on  the  video  monitor  along  with  a pair  of  electronically  generated 
crosshairs.  The  operator  identifies  the  location  of  the  peak  and  enters  a 
series  of  change  valves  until  the  crosshairs  and  peak  are  coincident.  This 
operation  establishes  the  first  correlation  peak  for  the  computer;  the  auto- 
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INITIALIZE  SCANNING 
SEQUENCE 


POSITION  GALVANOMETERS 
AT  FIRST  SCAN  POINT 


LOCATE  CORRELATION  PEAK 
ON  VIDEO  MONITOR 


INITIATE  AUTOMATIC  SCANNING 
AND  CORRELATION  TRACKING 


POSITION  GALVANOMETER 
AT  SCAN -POINT 


SAMPLE  LOCATION  AND 
AMPLITUDE  OF  PEAK 


STORE  PARALLAX  VALUES 


PREDICT  NEXT  APERTURE  ' 
FOR  CORRELATION  SEARCH 


DISPLAY/PRINT  PARALLAX  DATA 


Figure  5.  1.  3 Flowchart  for  Software  Program  Operation 
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matic  data  collection  process  now  begins. 


The  scanning  beam  is  directed  to  the  next  point.  The  location  of  a fixed 
aperture  region,  representing  approximately  1 per  cent  of  the  sensors  field 
of  view,  is  simultaneously  displayed  on  the  video  monitor.  This  aperture 
determines  the  region  of  the  active  video  frame  examined  for  the  correlat- 
ion. The  interface  electronics  search  this  region  for  the  most  intense  point 
and  store  its  location  and  amplitude  in  a series  of  buffers.  When  the  end  of 
the  aperture  has  been  reached,  a flag  is  set.  The  computers,  sensing  this 
flag,  reads  and  subsequently  stores  the  contents  of  the  buffers.  This  search 
and  store  sequence  is  repeated  as  required  by  the  valve  Nl.  The  series 
of  values  determined  for  the  point  are  appropriately  averaged  and  entered 
into  the  array  of  parallax  valves.  Based  on  the  last  locations  of  the  correl- 
ation peak,  the  location  of  the  aperture  for  the  next  scan  point  are  then 
collected  using  this  predicted  aperture  location. 

The  data  collection  process  is  repeated  until  the  last  scan  point  has 
been  reached.  The  programmable  control  unit  then  display  the  parallax 
data  collected  for  the  scan.  Two  types  of  display  are  used;  a video  display 
is  provided,  as  well  as  a point  out.  The  electronic  crosshairs  are  directed 
to  trace  the  profile  defined  by  the  correlation  peak  during  the  scan;  this 
trace  is  displayed  on  the  video  monitor.  The  array  of  valves  representing 
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the  horizontal  and  vertical  locations  and  the  correlation  intensity  are  printed 
on  the  output  terminal;  the  valves  can  be  used  for  further  analysis  or  data 
collection. 

The  software  program  controlling  the  data  collection  process  has  five 
major  subroutines.  These  subroutines  direct  or  control  a specific  portion 
of  the  overall  process.  The  GAL  VO  subroutine  controls  the  values  output 
to  the  scanning  system;  the  subroutine  accepts  two  digital  values,  directs 
the  digital-to -analog  conversions,  and  routes  these  analog  values  to  the 
galvanometer  drivers.  The  AP  subroutine  controls  the  location  and  size 
of  the  electronic  aperture.  The  AP  subroutine  accepts  four  arguments 
which  represent  the  locations  of  crosshairs  on  the  video  monitor  and  define 
the  active  search  region  for  the  peak  detection  electronics;  the  subroutine 
checks  the  bounds  on  these  arguments  and  injects  them  into  the  video  signal. 
The  remaining  three  subroutines  provide  the  actual  correlation  peak  acquis- 
ition. The  PEAK  subroutine  reads  the  contents  of  the  buffers  holding  the 
horizontal  location,  vertical  location,  and  intensity  of  the  correlation  peak. 

The  subroutine  also  provides  the  timing  and  control  to  ensure  synchro- 
nism of  the  scanning  process  with  the  video  detector.  The  PEAK  routine 
calls  the  two  additional  subroutines,  ADC  and  CRVT,  to  support  the  inter- 
face read  operation.  The  ADC  routine  directs  an  analog  to  digital  conver- 
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sion  of  the  peak  intensity,  whereas  the  CRVT  routine  masks  unnecessary 
bits  from  the  two  location  registers. 

I 

5.  2 IMF  SYSTEM:  DEVELOPMENT  RESULTS  AND  STATUS 

During  the  past  and  present  contractual  phases,  the  concepts  and  tech- 
niques developed  for  the  IMF  system  have  demonstrated  the  applicability 
of  coherent  optical  processing  to  the  stereocompilation  problem.  One  parti- 
cular area  in  which  a significant  improvement  in  correlation  has  been  ob- 
tained is  the  preprocessing  of  input  imagery.  The  edge  detail  contrast  of 
an  aerial  photograph  is  one  of  the  most  important  factors  affecting  correlat- 
ion performance  in  both  coherently  and  incoherently  illuminated  optical 
systems;  images  having  greater  edge  detail  contrast  typically  produce  cor- 
relation peaks  with  a higher  signal-to-noise  ratio  than  do  images  with  lower 
contrast.  High  contrast  regions  have  more  energy  in  the  spatial  frequency 
bands  critical  to  matched  filter  correlation  and  scatter  less  noise  from  the 
film  grain. 

Two  techniques  were  examined  which  demonstrated  successful  edge 
enhancement.  Silver-masked  input  scenes  prepared  manually  and  input 
scenes  prepared  on  automatic  dodging  contract  printing  equipment  showed 
an  improvement  in  correlation  performance  in  comparison  to  standard  con- 
tact points  onto  high  contrast  emulsions.  In  the  "silver  masking"  technique, 
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a silver  mask  is  formed  by  printing  the  original  diapositive  aerial  photo- 
graph onto  a low  contrast,  negative  working  film.  The  original  photograph 
is  then  placed  in  contact  with  an  unexposed  photographic  plate  having  a high 
contrast  emulsion.  The  silver  mask  is  placed  on  top  of  the  diapositive. 
During  exposure,  the  diapositive  forms  a sharp  positive  image  on  the  unex- 
posed plate,  while  the  silver  mask  casts  a blurred  negative  image.  On  a 
gross  scale,  the  transmissions  of  the  original  and  the  mask  tend  to  cancel 
and  produce  a nearly  uniform  exposure;  however,  on  a smaller  scale,  the 
sharp  edge  detail  of  the  diapositive  is  retained,  since  the  mask  produces  a 
blurred  image.  However,  gross  tonal  variations  are  smoothed  in  the  high 
contrast  copy,  and  high  edge  contrast  is  maintained  throughout. 

The  automatic  dodging  contact  printer  also  provides  edge  enhanced 
imagery  with  gross  tonal  variations  removed.  The  automatic  printer  uses 
a CRT  as  a light  source  with  additional  exposure  control  circuitry.  The 
scanning  spot  of  the  CRT  is  projected  onto  the  plane  containing  the  image  to 
be  copied  and  the  copy  material.  A photomultiplier  tube  (PMT)  is  used  to 
monitor  the  transmitted  light  from  the  CRT  through  the  image.  The  PMT 
output  signal,  a measure  of  local  density,  is  used  to  increase  or  decrease 
the  velocity  of  the  scanning  spot.  As  the  scanning  spot  is  swept  in  a raster 
pattern  across  the  input  scene,  the  copy  material  revieves  varying  point- 
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to-point  exposure  determined  by  local  density  values.  The  amount  of  edge 
enhancement  in  the  resulting  image  is  determined  by  the  size  of  the  scan- 
ning spot  and  contrast  of  the  copy  material. 


Although  the  silver  masking  technique  ultimately  offers  a greater  im- 
provement in  correlation  performance,  the  dodging  contact  printer  can  be 
adapted  to  automatic  operation.  In  addition,  further  modifications  to  the 
automatic  dodging  process  are  possible  which  can  provide  nearly  equivalent 
performance.  These  results  are  also  applicable  to  electronic  correlation, 
such  as  the  AS-  1 1 stereocompilation  system. 

The  accuracy  and  precision  provided  by  coherent  optical  processing 
techniques  have  also  been  demonstrated.  Parallax  data  collected  on  the 
IMF  system  was  reduced  and  compared  to  similar  data  collected  on  the 
^endix  AS-11  system.  The  data  reduction  accounted  for  the  scale  change, 
as  well  as  the  rotational  and  translational  biases  present  in  the  data.  The 
data  sets  obtained  are  shown  in  Figure  5.  2-1.  The  parallax  data  obtained 
with  both  systems  for  a section  of  the  Canadian  Test  Model  are  in  good 
agreement.  The  root  mean  square  difference  between  data  sets  is  approxi- 
mately 25  u m if  the  small  region  (located  near  14  mm)  at  the  edge  of  a 
precipitous  cliff  is  ignored.  (The  position  of  the  cliff  varies  rapidly  and  a 
slight  displacement  between  the  IMF  and  AS- 11  scan  paths  can  introduce 


large  parallax  changes).  For  typical  operational  parameters,  this  parallax 
difference  corresponds  to  a ground  profile  uncertainty  of  0.  4 m.  Consider- 
ing the  hardware  limitations  of  the  present  developmental  system,  these 
results  show  remarkable  accuracy  and  demonstrate  the  feasibility  of  employ- 
ing coherent  optical  techniques  for  stereocompilation. 

Methods  for  improving  the  accuracy  of  the  IMF  system  have  also  been 
examined.  In  particular,  the  performance  of  the  scanning  system  has  been 
investigated.  A major  result  is  the  improvement  in  beam  pointing  accuracy 
achievable  with  a closed  loop,  temperature  regulated  galvanometer.  This 
galvanometer  type  exhibited  a pointing  accuracy  approximately  five  times 
more  precise  than  that  of  the  corresponding  open-loop  devices  currently 
used  in  the  system.  Over  a three  hour  observation  period,  the  angular 
drift  of  the  open-loop  galvanometer  was  typically  1.  5 mrad,  corresponding 
to  a positioning  error  of  900  n m.  The  stability  of  the  closed-loop,  temper- 
ature regulated  galvanometers  was  considerably  better;  the  rms  error  in 
pointing  accuracy  over  a two  hour  period  was  0.  26  mrad.  The  residual 
errors  of  the  closed  loop  galvanometers  can  be  corrected  with  optical  feed- 
back  techniques.  The  current  phase  of  system  development  has  been  pri- 
marily concerned  with  three  areas:  the  identification  of  a solid  state  ima- 
ging device  for  correlation  detection,  the  implementation  of  an  electronic 
interface  for  automatic  correlation  location,  and  the  development  of  soft- 
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ware  for  parallax  collection  at  30  samples  per  second. 


Three  different  imaging  devices  were  evaluated  as  possible  output  de- 
tectors; a standard  silicon  vidicon,  a charge-coupled  device,  and  a charge 
injection  device.  A description  of  their  operation  and  the  details  of  their 
performance  are  given  in  Section  IV  of  this  report.  The  charge  injection 
device  was  the  candidate  detector  selected  for  the  IMF  system.  A CID 
camera  system  was  procured  and  subsequently  installed  in  the  breadboard. 
To  account  for  the  lower  sensitivity  of  the  CID  in  comparison  with  the  pre- 
vious vidicon  camera,  bleached  phase  matched  filters  were  used  for  cor- 
relation. These  filters  allowed  a significant  increase  in  diffracted  energy 
without  a decrease  in  signal-to-noise  ratio;  the  average  intensity  provided 
by  the  bleached  filters  to  form  the  correlation  was  sufficient  for  the  CID 
camera.  The  solid  state  imaging  device  also  exhibited  lower  video  noise 
levels  than  the  conventional  electron  beam  imaging  tubes  examined. 

An  electronic  interface  providing  the  required  processing  and  control 
functions  for  automatic  correlation  peak  detection  was  assembled.  This  in- 
terface system,  described  in  Section  5.  1.2,  is  compatible  with  both  the 
CID  solid  state  array  and  conventional  vidicon-type  devices.  The  accuracy 
and  precision  obtained  with  the  interface  were  evaluated  in  both  static  and 
dynamic  modes.  For  the  static  mode  tests,  a correlation  peak  was  dis- 
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played  on  the  video  monitor.  This  correlation  peak  had  an  adjustable  in- 
tensity and  was  the  only  signal  in  the  output  plane  of  the  optical  processor. 
Under  software  control,  the  interface  electronics  were  directed  to  re- 
peatedly acquire  the  position  of  the  isolated  peak.  During  this  process,  the 
aperture  which  restricts  the  sensor  field -of -view  was.  fixed  and  represen- 
ted approximately  one  per  cent  of  the  total  sensor  field.  Similarly,  the 
galvanometer  scanning  unit  was  maintained  at  a fixed  location. 


The  accuracy  of  the  interface  electronics  in  locating  the  defined  cor- 
relation peak  was  determined  for  several  intensity  levels  ranging  from  a 
visually  detectable  level  of  25  mV  to  an  intense  correlation  peak  of  200  mV. 
The  results  of  these  static  mode  tests  were  analysed  and  subsequently 
graphed  in  histogram  form.  These  histograms  are  shown  in  Figures  5.  2-2 
through  5.  2-4.  The  histograms  show  the  frequency  of  position  occurrence 
as  a function  of  position;  the  positions  have  been  normalised  to  span  a fif- 
teen unit  interval.  Each  unit  represents  a single  position  movement.  In 
the  vertical  direction,  the  increment  represents  one  clock  count.  In  the 
spatial  domain,  both  of  these  units  correspond  to  a nominal  25 n m displace- 
ment. Examination  of  the  histograms  show  that,  although  the  positional 
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TRACKING  ACCURACY  FOR  25  MV  INPUT  SIGNAL 


HORIZONTAL  POSITION  (RELATIVE  CLOCK  COUNT) 
a.  HORIZONTAL  TRACKING  ACCURACY 


VERTICAL  POSITION  (RELATIVE  LINE  NUMBER) 
b.  VERTICAL  TRACKING  ACCURACY 


c.  VIDEO  CORRELATION  SIGNAL 


Figure  5.  2-2  Tracking  Accuracy  and  Video  Signal  for  Input  Correlation 
Peak  of  25  mV 
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TRACKING  ACCURACY  FOR  100  MV  INPUT  SIGNAL 
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HORIZONTAL  POSITION  (RELATIVE  CLOCK  COUNT) 
a HORIZONTAL  TRACKING  ACCURACY 


VERTICAL  POSITION  (RELATIVE  LINE  NUMBER) 
b.  VERTICAL  TRACKING  ACCURACY 


c.  VIDEO  CORRELATION  SIGNAL 


Figure  5.  2-3  Tracking  Accuracy  and  Video  Signal  for  Input  Correlation 
Peak  of  100  mV 
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TRACKING  ACCURACY  FOR  200  MV  INPUT  SIGNAL 


HORIZONTAL  POSITION  (RELATIVE  CLOCK  COUNT) 
a.  HORIZONTAL  TRACKING  ACCURACY 


VERTICAL  POSITION  (RELATIVE  LINE  NUMBER) 
b.  VERTICAL  TRACKING  ACCURACY 


c.  VIDEO  CORRELATION  SIGNAL 


Figure  5.  2-4  Tracking  Accuracy  and  Video  Signal  for  Input  Correlation 
Peak  of  200  mV 
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The  test  of  the  interface  electronics  in  a dynamic  mode  required  the 
development  of  software  capable  of  data  collection  at  30  samples  per  second. 
Software  initially  developed  failed  to  reach  this  execution  rate.  However, 
refinements  and  modifications  to  the  controlling  software  program  were 
implemented  which  reduced  execution  time  to  acceptable  levels.  The  pri- 
mary change  was  the  use  of  integer  arithmetic  rather  than  the  normal  float- 
ing point  format.  In  this  scheme,  the  correlation  positional  valves,  as  well 
as  the  aperture  location  and -the  galvanometer  addressee,  are  pure  integer*. 
Similarly,  any  computation  or  calculation  involving  these  values  must  be 
performed  in  integer  format,  and  care  must  be  taken  to  ensure  that  proper 
results  are  obtained.  In  an  operational  sense,  the  integer  formats  are  not 
significant  drawbacks  because  any  conversions  to  the  floating  point  format 
can  be  accomplished  off-line  during  a period  when  speed  is  not  a paramount 
consideration.  The  final  software  program  developed  was  capable  of  data 
collection  at  TV  frame  rates  of  30  parallax  samples  per  second. 
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operation  of  these  features  generated  additional  noise  in  the  correlation  de- 
tection and  location  process.  Computer  clock  noise  was  coupled  into  the 
video  signal;  this  noise  was  reduced  to  acceptable  levels  by  suitable  low 
pass  filtering.  A ringing  phenomena  was  observed  in  dynamic  testing  on  the 
aperture  boundaries.  The  ringing  caused  an  uncertainty  in  the  precise  time 
and  location  of  the  aperture  boundaries.  However,  the  video  gating  cir- 
cuitry, operating  from  a fixed  clock  signal,  has  a predetermined  and  peri- 
odic window.  Bocauee  of  the  uncertainty  in  the  aperture  location,  the  gat- 
ing circuit  sometimes  allowed  undesired  signals  to  reach  the  peak  detector. 
The  undesired  signal  was  a portion  of  the  aperture  boundary.  In  regions 
where  weak  correlation  signals  were  obtained,  the  aperture  boundary 
(crosshairs)  exceeded  the  intensity  of  the  correlation  signal  and  was  erron- 
eously identified  as  the  actual  correlation  peak. 

The  software  controlling  program  used  a predictive  algorithm  to  cal- 
culate the  location  of  the  aperture.  The  incorrect  data  points  prevented 
proper  action  of  the  agorithm.  The  parallax  data  gathered  with  the  system 
in  a dynamic  mode  had  significant  variations  because  of  the  ringing  problem. 

Two  approaches  to  correct  this  issue  were  identified.  The  first  ap- 
proach involved  a refinement  of  the  hardware  design  to  take  advantage  of 
components  which  have  recently  become  available.  Integrated  circuits  to 
provide  the  gating  function  and  the  peak  detection  are  now  available 
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which  satisy  operational  requirements.  Only  a discrete  electronic  design 
was  possible  during  the  initial  program  phases.  The  substitution  of  the 
integrated  circuits  for  the  discrete  electronics  should  significantly  reduce 
the  noise  and  ringing  levels,  and  thus  improve  performance.  The  second 
approach  was  the  implementation  of  a completely  digital  peak  detection 
system.  Component  costs  and  cycle  times  for  this  approach  have  decreased 
significantly  since  original  program  funding.  The  digital  peak  detection  ap> 
p-roach  is  at  this  time  a major  candidate.  In  addition,  the  digital  peak  det- 
ection  approach  offers  a greater  potential  for  the  feature  extraction  process; 
the  approach  is  directly  compatible  with  the  digitizing  process  required  to 
obtain,  convert  and  manipulate  selected  features  of  an  image.  Further  pur- 
suit of  these  two  approaches  was  outside  the  scope  of  current  funding. 

The  technical  effort  for  the  program  was  terminated  prior  to  the  com- 
plete demonstration  of  system  goals.  A time  delay  was  partially  responsible 
for  a considerable  amount  of  labor  expended  to  repair  the  government  furnished 
galvanometer  scanning  system.  The  recurring  failures  of  the  galvanometer 
drivers,  critical  for  system  operation,  prevented  additional  development  of 
the  interface  electronics. 
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Despite  the  premature  termination  of  the  technical  activity,  all  of  the 
primary  stereocompilation  goals  were  partially  demonstrated;  these  in- 
clude: 

• Solid  state  area  sensor  arrays  were  surveyed  for  possible  use 
in  correlation  detection;  a charge-injection  device  was  selected 
and  subsequently  installed  in  the  experimental  system. 

e Interface  electronics  necessary  to  perform  automatic  peak 
detection  were  designed,  fabricated,  and  tested. 

e The  software  control  routines  required  to  drive  the  parallax 

collection  process  were  demonstrated  at  data  sample  rates  of  30 
frames  per  second. 
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SECTION  VI 

FEATURE  EXTRACTION 

The  identification  and  subsequent  location  of  patterns  from  aerial  pho- 
tographic imagery  provides  information  of  significant  value  to  both  govern- 
ment and  civilian  agencies.  Government  groups  gather  information  relating 
to  tactical  weapon  deployment  and  the  development  status  of  strategic  areas. 
Similarly,  civilian  agencies  routinely  monitor  earth  resources  and  perform 
survey  functions  using  aerial  photography.  Presently,  the  major  portion  of 
these  efforts  are  performed  manually  by  trained  photo -interpreters.  These 
interpreters,  relying  on  past  experience  and  developed  skills,  search  indi- 
vidual frames  of  imagery  to  locate  and  catalog  the  patterns  of  interest. 
Although  aided  partially  by  computer  data-logging  techniques,  this  location 
and  catalog  process  is  tedious  and  slow.  The  increasing  volume  of  imagery 
generated  for  analysis  cannot  be  processed  in  a timely  manner  with  manual 
techniques.  To  satisfy  the  critical  demand  for  current  and  reliable  infor- 
mation, an  automated  pattern  recognition  scheme  must  be  developed. 

Whether  performed  manually  or  automatically,  the  actual  pattern 
recognition  process  involves  three  key  steps.  First,  the  features  necessary 
to  characterize  and  distinquish  particular  patterns  must  be  determined. 
Second,  these  selected  features  must  be  extracted  for  the  different  areas 
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◦f  the  image.  Third,  the  relevant  data  must  be  processed  to  classify  or 
recognize  the  patterns  of  interest.  These  steps  of  feature  selection,  ex- 
traction, and  classification  must  take  into  account  variations  in  pattern 
appearance  (e.  g. , shape,  orientation,  and  scale)  expected  statistically 
both  vtrithin  a frame  and  between  frames  of  aerial  imagery;  this  capability 
is  a major  consideration  for  any  pattern  recognition  process.  The  photo- 
interpreter relies  on  his  experience  and  judgement  in  handling  these 
variations;  the  automated  process  must  possess  a comparable  capability. 

The  pattern  recognition  process  can  range  from  relatively  simple  tasks, 
such  as  separating  urban  and  rural  regions,  to  highly  complex  tasks,  such 
as  identifying  aircraft  types  present  at  landing  facilities.  Similarly,  the 
features  associated  with  these  tasks  can  vary  over  a wide  spectrum.  The 
selection  of  features  to  perform  these  task*  is  often  done  in  an  intuitive  or 
subjective  fashion.  A set  of  features  which  are  felt  to  provide  a separabi- 
lity between  patterns  or  classes  can  be  developed  and  subsequently  tested. 
The  effectiveness  and  completeness  of  the  feature  set  are  then  evaluated  in 
initial  pattern  recognition  or  class  separation  tests.  Based  on  this  evalua- 
tion, the  original  set  of  features  can  be  modified  to  provide  better  per- 
formance. For  example,  a feature  with  little  Influence  on  the  process 


might  be  deleted,  while  another  feature,  not  initially  included  in  the 

feature  set,  might  be  added  to  provide  a higher  discrimination  measure.  In 
addition,  various  features  can  be  altered  to  form  linear  and  non-linear  com- 
binations which  offer  simplified  processing  or  faster  sorting  time.  This 
feature  refinement  process  can  be  continued  until  satisfactory  performance 

is  achieved. 


The  selection  of  a set  of  features  is  a critical  stage  of  the  pattern 
recognition  process.  This  feature  set  must  be  complete;  that  is,  the  fea- 
tures must  adequately  describe  the  patterns  of  interest  and  differentiate 
among  them.  The  features  must  also  be  efficient  to  provide  a maximum 
accuracy  with  minimum  processing  storage  and  time  demands.  In  addition, 
the  features  must  be  relatively  insensitive  to  the  statistical  variations  in 
pattern  appearance. 

The  major  aim  of  the  feature  selection  process  is  to  choose  the  minimum 
number  of  features  which  preserve  class  separability.  The  effectiveness  of 
the  chosen  feature  set  can  be  measured  by  the  probability  of  error.  This 
probability,  affected  by  the  scatter  and  variation  encountered,  is  highly 
dependent  on  the  particular  recognition  task  at  hand. 

Pattern  recognition  from  aerial  photographs  with  a purely  digital 
processing  facility  must  rely  primarily  on  a high  resolution  digital  pixel 
representation  of  the  image.  From  this  representation  other  features  can 
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be  constructed.  A low  resolution  pixel  matrix  can  be  developed  for  course 
recognition  through  averaging  techniques.  Various  transforms,  including 
the  Fourier  and  Mellon  transforms,  can  be  calculated  for  different  size 
regions  of  the  image.  The  cross-correlation,  yielding  the  parallax  values 
for  ground  profiles,  can  be  computed  for  stereo  pairs  of  images.  Low, 
high,  and  bandpass  spatial  filtering  of  the  original  pixel  representation 
can  be  implemented  through  suitable  software  algorithms.  However,  all  of 
these  representations  demand  a significant  amount  of  memory  storage  and 
require  additional  processing  time. 

An  alternative  or  adjunct  to  scanning  and  digitally  processing  an  image 
for  feature  extraction  is  the  use  of  optical  techniques  directly  on  the  image. 

A coherent  optical  processing  system  has  considerable  potential  in  the 
feature  extraction  area.  An  optical  processing  system,  such  as  the  IMF 
breadboard,  provides  direct  access  to  the  input  imagery,  the  Fourier  trans- 
form of  that  imagery,  and  a filtered  version  of  the  image.  The  simultan- 
eous presence  of  these  three  elements  in  a single  system  is  a unique 
situation  having  significant  advantages  for  pattern  recognition.  In  addition, 
these  functions  can  be  provided  without  a demand  for  digital  memory  storage 
or  an  increase  in  processing  times;  these  are  key  considerations  for 
feature  extraction  and  subsequent  pattern  recognition. 
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A coherent  optical  processor  can  directly  yield  the  digital  pixel  repre- 
sentation at  various  scales  for  aerial  photographs.  A simple  modification 
to  the  scanning  geometry  of  the  IMF  system  would  provide  this  capability. 
To  accomplish  this  task,  a set  of  lenses,  mounted  in  a turret  assembly 
whose  position  is  computer  controlled,  is  added  to  the  scanning  beam 
optics.  Each  lens  is  chosen  to  provide  a different  diameter  scanning  beam 
in  the  input  plane  of  the  processor.  A single  detector  in  the  transform 
would  collect  all  the  light  transmitted  through  the  photograph;  a 
single  analog-to-digital  converter  sampling  the  detected  energy  would 
provide  the  pixel  valve  for  the  given  location  and  area  of  the  scanning  beam. 
By  automatically  selecting  different  lenses  of  the  turret  assembly,  the 
computer  could  digitise  the  image  at  various  resolutions.  In  addition, 
simple  spatial  filtering  could  be  employed  to  obtain  filtered  versions  of  the 
imagery.  For  example,  a transmissive  or  opaque  disc  located  prior  to  the 
detector  in  the  transform  plane  would  allow  a low  or  high  passed  filtered 
image  to  be  digitized.  Similarly,  a transmissive  annular  filter  in  front  of 
the  detector  would  provide  a spatially  band-passed  image  to  be  digitized. 
The  image  could  thus  be  digitized  in  a variety  of  formats  with  a coherent 
processing  system.  This  sampling  process  is  capable  of  high  speeds;  a 
digitizing  rate  of  50K  pixels  per  second  at  500  lpi  resolution  is  feasible. 

Power  spectral  analysis  can  also  be  directly  implemented  with  the  IMF 
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system.  The  transform  plane  of  the  IMF  system  contains  a light  intensity 
distribution  which  is  the  spatial  frequency  power  spectra  of  the  image  in  the 
input  plane.  The  measurement  of  the  intensity  distribution  can  be  simply 
accomplished  by  locating  a two  dimensional  solid-state  area  array  in  the 
transform  plane;  analog -to-digital  conversion  of  the  resulting  signal 
provides  a digital  representation  of  the  power  spectra  at  video  bandwidths. 
The  power  spectra  can  thus  be  measured  directly  at  high  rates  instead  of 
calculated  or  computed  from  digital  pixel  representations.  In  addition, 
since  the  scanning  system  can  access  different  locations  of  the  input  image 
and  provide  a range  of  scanning  spot  diameters,  the  power  spectra  can  be 
determined  directly  for  localized  regions  of  the  image.  Thus,  changes  in 
the  spectra  for  various  local  regions  could  be  detected  and  utilized  for  fea- 
ture extraction. 

A conventional  two  dimensional  array,  such  as  a C ID  used  in  conjunc- 
tion  with  an  a/d  converter  and  a digital  computer  can  be  used  to  synthesize 
unique,  specially  shaped  apertures  in  the  transform  plane.  For  instance,  a 
series  of  concentric  rings  and  radial  wedges  (similar  to  those  of  the  Recog- 
nition System  ROSA  detector)  can  be  simulated.  Although  this  pattern  is 
simplistic,  successful  photograph  sorting  has  been  demonstrated  in  relati- 
vely simple  applications.  The  value  of  the  ring  and  wedge  detector  pattern 
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for  feature  extraction  requires  further  investigation. 


The  IMF  system  can  provide  another  feature  measure  for  pattern  re- 
cognition; parallax  can  be  determined  for  local  areas  of  the  image.  The 
parallax  valve  can  discriminate  certain  classes  of  terrain.  For  example, 
the  frequent  and  large  parallax  changes  characterizes  some  terrain  regions 
(such  as  mountains  and  urban  areas),  but  not  others  (such  as  agricultural 
regions  and  bodies  of  water).  Parallax  measurements  have  potential  as  an 
effective  feature  for  pattern  recognition. 


A series  of  experiments  were  conducted  to  demonstrate  the  prelimin- 
ary value  of  a coherent  optical  processing  system  for  feature  extraction  and 
pattern  recognition.  Conventional  contact  copying  techniques  were  used  to 
generate  sample  imagery  on  high  resolution  plates  from  typical  aerial  pho- 
tographs. The  processed  images  were  placed  in  the  input  plane  of  the  IMF 
system,  and  their  power  spectra  were  observed.  For  the  experiments,  a 
scanning  beam  diameter  of  5 mm  was  used  for  illumination  and  a DC  block 
was  located  in  the  transform  plane  to  eliminate  the  low  frequency  coxnpon- 


Sections  of  different  terrain  regions  and  the  corresponding  power 
spectra  are  shown  in  Figures  6-1  through  6-3.  Three  different  terrain 
types  were  selected  which  include  an  urban  area,  an  airway  landing  faci- 
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a.  SECTION  OF  ORIGINAL  AERIAL  PHOTOGRAPH 


2 CY/mm 


b.  CORRESPONDING  POWER  SPECTRUM 
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Figure  6-1  Image  and  Power  Spectrum  of  Typical  Urban  Area 
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a.  SECTION  OF  ORIGINAL  AERIAL  PHOTOGRAPH 
2 CY/mm 


b.  CORRESPONDING  POWER  SPECTRUM 
Figure  6-2  Image  and  Power  Spectra  of  Airway  Landing  Facility 


a.  SECTION  OF  ORIGINAL  AERIAL  PHOTOGRAPH 

2 CY/mm 


b.  CORRESPONDING  POWER  SPECTRUM 
Figure  6-3  Image  and  Power  Spectra  of  Rural  Terrain 


lity,  and  a rural  region.  The  urban  region,  shown  in  Figure  6-1  (a),  con- 
tains a housing  development  with  well-defined  features;  the  houses  and 
streets  form  a two  dimensional  pattern  with  a reasonably  pieriodic  struct- 
ure. This  regularity  is  evidenced  in  the  power  spectra  shown  in  Figure 
6-1  (b)  by  the  cross-like  transform  pattern  is  caused  by  the  regular  edges 
and  even  features  of  the  housing  development.  A similar  result  occurs 
for  the  airway  landing  facility  shown  in  Figure  6 -2(a).  The  sharp  edges 
and  constant  orientation  of  the  runway  generate  a FourieT  transform  of 
Figure  6-2  (b)  having  significant  energy  in  only  one  direction. 

Fine  scale,  continued  features  of  an  image  are  represented  in  the 
transform  plane  by  an  intensity  pattern  perpendicular  to  the  feature  direct- 
ion. Typically,  man-made  objects  are  characterized  by  fine-scale, 
continued  features  which  have  regular  structure.  Examples  include  roads, 
housing  developments,  and  urban  centers.  The  detection  of  preferred  or 
high  intensity  axes  in  the  transform  plane  indicate  structure  associated 
with  regular  characteristics  and  yield  information  on  their  orientation. 

For  comparison,  a rural  region  and  its  power  spectrum  are  shown  in 
Figure  6-3.  The  rural  region  shown  in  Figure  6-3  (a)  is  devoid  of  man- 
made structures  and  exhibits  no  regular  or  periodic  structure.  The 
associated  power  spectra  of  figure  6-3  (b)  similarly  has  no  preferred  dir- 
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ection  or  characteristics.  The  intensity  of  the  transform  decreases  in  a 
uniform  radial  fashion. 

Additional  investigation  of  the  feature  extraction  and  pattern  recogni- 
tion capabilities  of  coherent  optical  processing  is  warranted.  The  three 
figures  demonstrate  the  effectiveness  of  a simple  feature,  the  directional 
dependence  of  the  Fourier  transform,  in  separating  or  sorting  man-made 
and  rural  terrain  regions.  With  immediate  access  to  the  Fourier  trans- 
form, a coherent  processing  system  could  perform  the  separation  simply, 
directly  and  accurately.  The  results  of  the  separation  operation  could  be 
used  to  pre-screen  imagery  for  subsequent  manual  or  digital  classification 
or  to  provide  initial  estimates  of  terrain  boundaries  for  additional  optical 
recognition. 

While  coherent  optical  processing  for  feature  extraction  is  promising, 
there  is  no  guarantee  that  this  technique  will  be  highly  successful  and,  in 
any  case,  a fairly  significant  technical  effort  will  be  required  to  develop 
and  prove  an  operational  system.  On  the  other  hand,  the  same  comments 
can  be  applied  to  digital  processing  of  scanned  imagery.  There  certainly 
is  hope  that  truly  competent  systems  for  feature  extraction  and  classifica- 
tion can  be  developed;  coherent  optical  processing  could  be  a significant 
portion  of  this  system.  However,  the  problem  is  one  of  great  difficulty  and 
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require*  the  establishment  of  a long  range,  well  directed  Drosram. 


APPENDIX  A 

CIRCUIT  SCHEMATICS  FOR  IMF 
INTERFACE  ELECTRONICS 

This  appendix  details  the  components  and  connections  utilized  in 
the  IMF  interface  electronics.  The  appendix  includes  the  circuit  diagrams 
for  the  computer  interface,  the  aperture  location  module,  and  the  track/ 
hold  circuits.  In  addition,  the  analog  electronic  circuit  which  performs 
the  peak  detection  is  included.  A full  description  of  system  operation  is 
presented  in  Section  5.  1.  2. 
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APPENDIX  B 

This  appendix  contains  the  program  listings  for  the  software 
developed  for  the  IMF  interface.  The  software  controlling  programs  are 
written  in  Fortran-II  and  SABR,  compatible  with  the  DEC  OS-8  opera- 
ting system.  These  two  languages  were  selected  because  of  their  relative 
programming  simplicity  and  their  interfacing  convenience.  The  interface 
options  support  the  drive  and  control  of  the  galvanometer  scanner,  the 
selection  and  operation  of  the  peak  detection  module,  and  the  processing 
and  storage  for  parallax  data  collection.  The  flowchart  for  system  oper- 
ation and  the  description  of  program  functions  are  included  in  Section 
5.1.3. 
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AP  ERROR I L»  XIV  R*  349  T- 
L/R  TOO  SMALL 
AP“C 
.R  CDIT 

SSCAN.FT<8CAN.FT 


•L 

caBaBsaaaaasaaaaaaaaaasaaaasssaaaaaaasasaasaaaaaaaaaaaaaaaaaaaaa 

C VARIABLE  DEFINITION 
C 

C VARIABLES  FOR  ARERTURE  CONTROL  (AR  SUBROUTINE) 

C 1X1  — DIGITAL  VALUE  FOR  LEFT  HAIR 

C 1X2— DIGITAL  VALUE  FOR  RIGHT  HAIR 

C IT1— DIGITAL  VALUE  FOR  TOR  HAIR 

C IV2— DIGITAL  VALUE  FOR  BOTTOM  HAIR 

C IX  — TEMPORARY  USE  FOR  INCREMENT  ON  X POSITION 

C —RUNNING  AVERAGE  FOR  PAST  FIVE  X PEAK  LOCATIONS 

C IV  —TEMPORARY  USE  FOR  INCREMENT  ON  Y POSITION 

C —RUNNING  USE  FOR  INCREMENT  ON  Y POSITION 

C 

C INTEGER  VARIABLES  FOR  OALVO  CONTROL  < GAL VO  SUBROUTINE) 

C . _ IOX— VALUE  FOR  X POSITION 

C IOY— VALUE  FOR  Y POSITION 

C H —VALUE  INDEX  FOR  SCANNING  (300.1300.4) 

C N —VALUE  INDEX  FOR  PARALLAX  ARRAY 

C 

C INTEGER  VALUES  FOR  PEAK  LOCATION  (PEAK  SUBROUTINE) 

C JXL— X POSITION  OF  CORRELATION  PEAK 

C JYL— Y POSITION  OF  CORRELATION  PEAK 

C JAL— AMPLITUDE  OF  CORRELATION  PEAK 

C JXA<N>—  ARRAY  OF  X POSITIONS  OF  PEAK 

C JYA(N>— ARRAY  OF  Y POSITIONS  OF  PEAK 

C JANP ( N > -ARRAY  OF  PEAK  AMPLITUDES 

caaaaaaaasaaaaaaaaaa*aaass«a«aasaasaa*B**WBBS«SBSSBB«**BBBSB*SB 

C THIS  IS  THE  MAIN  PROGRAM  AREA 

DIMENSION  JXA< 201 > . JYA< 201 > , JAAP <201 > 

READ  (1.49)  Ml 

49  FORMAT (24HNOHBER  OF  SAMPLES/PIXEL-. 13) 

1X1-193 

. 1X2-193  

IY1-117 

IV2-117 

• JUS  AP  /SET  UP  CROSSHAIRS 

I0X-300 

IGY-300 

S JMS  OALVO  /MOVE  OALVO  ... 

C THI8  INITIALIZES  THE  APERTURE  FOR  THE  SCAN  1 1 1 

1 READ< 1 >2)  IX 

READ!  1 #2)  IV  . - 

2 FORMAT! 14 > 

IX1-IXUIX 

IX2-IX2+IX  . ...  

IV1-IV14IY 

IY2-IY2+IV 

S _ JMS  AP  /SET  UP  NEW  APERTURE 

WRITE < 1 >8)  IX1.IY1 

S FORMAT  < 4HLEFT » I4»4H  TOP. 14) 

READ! 1.3)  ENUF  
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3 FORMAT (4HCNUF.F3.1) 

IF  !ENUF>4.1.1 

4 CONTINUE 
IX-IXI 
IY-IY1 
IX1-IX-13 
IX2-IX+13 
IY1-IY-13 
IY2-IY413 

s jn8  ap  /*rr  up  aperture 

3 CONTINUE 

DO  20  M*300.1S00.4 

N-M/4-4F 

IOX-H 

IOY-M 

DO  10  NF-1.N1 

8 JM8  OALUO  /MOVE  QALVO  TO  SCAN  FOINT 

8 — JM8  FEAK  /SAMPLE  LOCATION  AND  PEAK  HCIBHT 

JXA!N)-JXA!N)+JXL 
JYA!N>-JVA!N)+JVL 
_ JAMF<N)-JAL 

10  CONTINUE 

JXA!N>-JXA!N>/N1 
. JYA!N)"JYA!N)/N1 

IF  (N-S)  13.13.12 

12  IX-!JXA!N-4>+JXA!M-3>+JXA!N-2>+JXA<N-l>+JXA<N>>/9 

. . ..  IY»! JYA! N-4)+ JYA!M-3>4JYA!N-2>4JVAIN-1 )4JYA!N>  >/l 

13  IX1-IX-I3 
1X2*1X413 

..  IV1-IV-13  . 

IY2-IV413 

8 JM8  AF  /SET  UF  INITIAL  AFERTURC 

20  CONTINUE 

WRITE  (1.30) 

30  FORMAT UX.SHINDEX.4X.5HXP0SN.4X.9MYP08N.4X.SHAHPTD) 

. DO  30  N-1.201 

WRITE! 1.43)N.JXA<N>.JYA(N>.JAMF<N> 

43  FORMAT! AX. IS. AX. 13 .AX. 13. AX. 19 > 

IX1*JXA!N> 

IX2-JXA!N> 

IV1-JYA1N) 

IV2-JYA!N) 

8 JM8  AF  /TRACE  THE  PARALLAX  PATH 

30  CONTINUE 

00  TO  800 


C THIS  8U8R0UTINC  IS  USED  TO  CONTROL  THE  OUTPUTS  TO  THE 
C TWO  SALVO  DRIVERS  IN  THE  ACME  SYSTEM.  THE  TWO  INTEBER 
C ARGUMENTS  AND  THEIR  RANGES  ARE! 

C 0 <IOX!X  GAL  VO  DRIVER  X204A 

C • <I9Y!V  SALVO  DRIVER) <2044 

C THE  IMPORTANT  INTERFACE  COMMANDS  FOR  THE  ROUTINE  ARE! 

C *331— -OUTPUT  X GAL  VO  VALUE 

C 4332— OUTPUT  Y GAL  VO  VALUE 

C THE  VALUES. WHICH  ARE  PRESENT  IN  THE  ACCUMULATOR. ARC 
C D/A  CONVERTED  AND  ROUTED  TO  THE  SALVO  DRIVERS  WHEN  THE 
C OUTPUT  COMMANDS  ARC  ENCOUNTERED. 

8 SALVO.  0000 

8 CLA 

8 TAD  MSX 

S 4331 

S TAD  MOV 

• 4332 

S ...  JNP.  4 NALUa. 


/REHCHSER  CALL I NO  ADDRESS 
/PREPARE  FOR  SHOVE 
/LOAD  X VALUE 
/SHOVE  IT 
/LOAD  V VALUE 
/SHOVE  IT 


C THIS 

IS  A SUBROUTINE 

TO  READ  THE  ACHE  INTERFACE.  THE 

C ARGUMENTS  ARE! 

C 

JXL  — 

THE  X PEAK  LOCATION 

C 

JYL  “ 

THE  Y PEAK  LOCATION 

C 

JAL  — 

THE  PEAK  AMPLITUDE 

C THE  KEY  INTERFACE  COMMANDS  ARE! 

C 

*341  — READ  X LOCATION 

C 

*342  — 

• READ  Y LOCATION 

S PEAK. 

0000 

/REMEMBER  CALLING  ADDRESS 

C MUST 

READ  THE  INTERFACE  LOCATION 

JXL-0 

JYL-0 

JAL-0 

310 

CONTINUE 

S 

CLA 

/CLEAR  AC 

s 

*314 

/RESET  FLAO 

S SYNC. 

*324 

/SKIP  WHEN  FLAG  SET 

s 

JHP  SYNC 

/WAIT  LOOP 

s 

JHS  ADC 

/READ  AMP  BEFORE  DROOP 

s 

DCA  \ JAL 

/STORE  PEAK  AMPLITUDE 

s 

NOP 

/NOW  FIND  THE  POSITIONS 

s 

*341 

/READ  X PEAK  LOCATION 

S Tl. 

JMS  CURT 

/MASK  OFF  TWO  MSB . COMPLEMENT 

s 

DCA  \JXL 

/STORE  THE  X PEAK  LOCATION 

s 

*342 

/READ  Y PEAK  LOCATION 

S T2. 

JMS  CVRT 

/MASK. COMPLEMENT 

S 

DCA  \JYL 

/STORE  THE  Y PEAK  LOCATION 

IF  <312- JYL)  313.313.320 

313 

JYL-JYL-312 
JAL— JAL 

320 

CONTINUE 

IF  <JXL -390)323. 310. 310 

323 

IF  <JYL-234>330.310.310 

330 

CONTINUE 

8 . JHP  I PEAK  /GO  MCK  TO  CALLING  PROM  AM 


C THIS  IS  THE  CONVERT  SUBROUTINE  PON  READING  THE  PEAK 

C LOCATIONS.  THE  TOP  TWO  BITS  MUST  BE  MASKED  OFF.AND 

C THE  REMA1NINO  WORD  MUST  DE  COMPLEMENTED.  THE  PEAK 

C LOCATION  IS  IK  THE  ACCUMULATOR. 

S CVRT*  OOOO  /REMEMBER  CALLING  ADDRESS 

S CMA  /COMPLEMENT 

S AND  <1777  /STRIP  IT 

S ..  JHP  I CVRT  /RETURN 


C THIS  IS  A ROUTINE  TO  READ  THE  VALUE  PROM  THE  ADG  CONVERTER 
C THE  IMPORTANT  CODE  COMMANDS  ARE  I 
C *330— CLEAR  1HE  A/D  REGISTER 

C ...  *331—  LOAD  THE  A/D  MUX 

C *332— START  THE  A/D  CONVERSION 

C *333— READ  THE  A/D  REGISTER 

C *334— SKIP  ON  CONVERSION  COMPLETION 

C A.YSA— LOAD  ENABLE  REGISTER 

C THE  MUX  CHANNEL  USED  TO  SAMPLE  THE  PEAK  IS  0900. 

C GET  ON  WITH  THE  PROGRAM ■ 


t 


9 TAP  (32 

9 *356 

9 6351 

9 6352 

9 DONE.  6354 
9 JHP  DONE 

9 6353 

S JHP  I AOC 


/SET  FOR  NO  INC. NO  INT  .UN  I POL  .NO  TNT 
/LOAD  ENASLE  RESISTED  WITH  0000 
/LOAD  MUX  WITH  0000  P OR  CHANNEL  0 
/START  CONVERSION 
/SKIP  IF  A/0  DONE 
/DOODLE  I 

/READ  A/D  VALUE  INTO  AC 
/RETURN 


C THIS  IS  A PROGRAH  TO  PROVIDE  CROSSHAIRS  AND  APERTURE  CONHANM 
C FOR  THE  ACHE  INTERFACE.  THE  FOUR  AROUHCNTO  ANSI 
C 1X1  —LEFT  HAIR 

C 1X2  —RIGHT  HAIR 

C IY1  —TOP  HAIR 

C IY2  — BOTTOH  HAIR 

C THE  IHPORTANT  CODE  COHHANDS  FOR  THE  INTERFACE  ARE! 

C 6511  —OUTPUT  LEFT  CROSSHAIR 

C 6312  —OUTPUT  RIOHT  CROSSHAIR 

C 6321  —OUTPUT  TOP  CROSSHAIR 

C 6322  —OUTPUT  SOTTOH  CROSSHAIR 

C 6314  —RESET  FRAHE  FLA8  FOR  SURE  SYNC 

C 6324  —SKIP  WHEN  FLAG  SET 

C ALL  ADDRESSES  ARE  LOADED  FROH  THE  ACCUMULATOR. 

C THE  RANGE  OF  VALUES  FOR  THESE  VALUN  ANSI 
C 1 < LEFT  < RIOHT  < 390 

C 1 <TOP  < SOTTOH  < 254 


9 AP,  0000  /RENEHSER  CALLING  AB8RC8S 

C CHECK  ON  HAIR  SOUNDS. 


410  IF  (IX2-IX1)  500.420.420 

420  IF  (IY2-IYI)  510.430.430 

430  IF  <1X1-1 > 520.440.440 

440  . ..  IF  <390-1X2)  530.450.450 

450  IF  < I Vl-1 > 540.460.460 

460  IF  <2S4-IV2>  550.470.470 

470  - CONTINUE 

S CLA 

S TAD  MX1  /LOAD  LEFT  HAIR 

S . . . 6311 -/SHOVE-  IT  OUT- 

TAD  \IX2  /LOAD  RIOHT  HAIR 

• A212  /SHOVE  IT 

0 TAD  MYl  /LOAD  TOP  HAIR 

• 0321  /SHOVE  IT 

• TAD  \IY2  /LOAD  SOTTOH  HAIR 

S _ 6322  /SHOVE  IT  

• * OP  /RETURN  TO  CALLINO  PROORAH 

500  WRITE  < 1 r SOS > 

505  _ FORMAT! 15HL/RWR0N0  ORDER) 


GO  TO  560 

510  WRITE  < 1.515) 

SIS  FORMAT < 15HT/S  WRONG  ORDER)  . 


GO  TO  360 

520 

WRITE  < 1.325) 

525 

FORMAT  < 13HL/R 
GO  TO  560 

TOO 

SHALL) 

530 

WRITE  < 1.535) 

535 

FORMAT  < 13HL/R 
GO  TO  560 

TOO 

LAROE) 

340 

WRITE  (1.545) 

545 

FORMAT  < 13HT/S 
GO  TO  560 

TOO 

SHALL) 

550 

WRITE  <1.555) 

• 

* 

355 

FORMAT! 13MT/S 

TOO 

LARGE) 

560 

WRITE  <1.570)1X1. IX2.IY1.IY2 

570 

FORMAT! 12HAP  ERRORS  L-.I7.4X.2HR-.17.4X.2HTA.I7.4X* 

S 

JHP  I AP 

/RETURN 

>»I7) 


